Extrapolating from lens design databases using deep learning

Côté, Geoffroi; Lalonde, Jean‐François; Thibault, Simon

doi:10.1364/oe.27.028279

Cited by 34 publications

(16 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, the optimization of adaptive optics systems has subject of deep learning applications (see [100] for a recent review). Many works are exploring the design of optical element configurations with deep neural networks and similarly flexible algorithms [101][102][103][104][105]. Co-design of instruments and observational strategies are being performed to optimize black hole observations [106].…”

Section: Experiments and Instrument Designmentioning

confidence: 99%

Machine Learning and Cosmology

Dvorkin,

Mishra-Sharma,

Nord

et al. 2022

Preprint

View full text Add to dashboard Cite

Methods based on machine learning have recently made substantial inroads in many corners of cosmology. Through this process, new computational tools, new perspectives on data collection, model development, analysis, and discovery, as well as new communities and educational pathways have emerged. Despite rapid progress, substantial potential at the intersection of cosmology and machine learning remains untapped. In this white paper, we summarize current and ongoing developments relating to the application of machine learning within cosmology and provide a set of recommendations aimed at maximizing the scientific impact of these burgeoning tools over the coming decade through both technical development as well as the fostering of emerging communities.

show abstract

Section: Experiments and Instrument Designmentioning

confidence: 99%

Machine Learning and Cosmology

Dvorkin,

Mishra-Sharma,

Nord

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Those approaches have been demonstrated to obtain state-of-the-art results in many computational photography tasks but not take one step further to optimize the optics together. G. Côté et al [2019; utilize deep learning to get lens design databases to produce highquality starting points from various optical specifications. However, they generally focused on the design of starting points, and the designing space is limited to spherical surfaces.…”

Section: Related Workmentioning

confidence: 99%

End-to-end complex lens design with differentiate ray tracing

Sun

Wang

et al. 2021

ACM Trans. Graph.

View full text Add to dashboard Cite

Imaging systems have long been designed in separated steps: experience-driven optical design followed by sophisticated image processing. Although recent advances in computational imaging aim to bridge the gap in an end-to-end fashion, the image formation models used in these approaches have been quite simplistic, built either on simple wave optics models such as Fourier transform, or on similar paraxial models. Such models only support the optimization of a single lens surface, which limits the achievable image quality. To overcome these challenges, we propose a general end-to-end complex lens design framework enabled by a differentiable ray tracing image formation model. Specifically, our model relies on the differentiable ray tracing rendering engine to render optical images in the full field by taking into account all on/off-axis aberrations governed by the theory of geometric optics. Our design pipeline can jointly optimize the lens module and the image reconstruction network for a specific imaging task. We demonstrate the effectiveness of the proposed method on two typical applications, including large field-of-view imaging and extended depth-of-field imaging. Both simulation and experimental results show superior image quality compared with conventional lens designs. Our framework offers a competitive alternative for the design of modern imaging systems.

show abstract

“…Together with gradient-based optimization, the obtained derivatives provide a searching direction in the hyper-parameter space to locally guide evolution of current design, improving performance in terms of the error metric. Combined with deep neural networks, such a differentiable engine could be employed for generating lens designs [33], [34] or image restoration [35], [36]. Recent works [13], [14] rely on differentiable ray tracing for end-to-end designs in computational imaging.…”

Section: Introductionmentioning

confidence: 99%

“…This memory issue has not been fully addressed in previous derivative-aware ray tracing works [29], [30], [33], [13] due to different application-oriented goals. In [13], the implementation of a differentiable ray tracing scheme enables image rendering, and thus rays are traced backwardly starting from the sensor plane, through the lenses, landing at the scene.…”

Section: Introductionmentioning

confidence: 99%

dO: A Differentiable Engine for Deep Lens Design of Computational Imaging Systems

Wang

Heidrich

2022

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

Computational imaging systems algorithmically post-process acquisition images either to reveal physical quantities of interest or to increase image quality, e.g., deblurring. Designing a computational imaging system requires co-design of optics and algorithms, and recently Deep Lens systems have been proposed in which both components are end-to-end designed using data-driven end-to-end training. However, progress on this exciting concept has so far been hampered by the lack of differentiable forward simulations for complex optical design spaces.Here, we introduce dO (DiffOptics) to provide derivative insights into the design pipeline to chain variable parameters and their gradients to an error metric through differential ray tracing. However, straightforward back-propagation of many millions of rays requires unaffordable device memory, and is not resolved by prior works. dO alleviates this issue using two customized memory-efficient techniques: differentiable raysurface intersection and adjoint back-propagation. Broad application examples demonstrate the versatility and flexibility of dO, including classical lens designs in asphere, double-Gauss, and freeform, reverse engineering for metrology, and joint designs of optics-network in computational imaging applications. We believe dO enables a radically new approach to computational imaging system designs and relevant research domains.

show abstract

Extrapolating from lens design databases using deep learning

Cited by 34 publications

References 6 publications

Machine Learning and Cosmology

Machine Learning and Cosmology

End-to-end complex lens design with differentiate ray tracing

dO: A Differentiable Engine for Deep Lens Design of Computational Imaging Systems

Contact Info

Product

Resources

About