Curriculum Learning for ab initio Deep Learned Refractive Optics

Abstract

Deep lens optimization has recently emerged as a new paradigm for designing computational imaging systems, however it has been limited to either simple optical systems consisting of a single element such as a diffractive optical element (DOE) or metalens, or the fine-tuning of compound lenses from good initial designs. Here we present a deep lens design method based on curriculum learning, which is able to learn optical designs of compound lenses ab initio from randomly initialized surfaces without human intervention, therefore overcoming the need for a good initial design. We demonstrate this approach with the fullyautomatic design of an extended depth-of-field computational camera in a cellphone-style form factor, highly aspherical surfaces, and a short back focal length.

Aberration-Aware Depth-From-Focus

“…We use the same depth range for the focus distance. We employ the DeepLens framework [30], [31] for ray tracing computation. The training process is run on a single A100 80G GPU and completed in approximately 6 hours.…”

“…Our AAT method consists of a lightweight point spread function (PSF) network and a re-rendering process to simulate aberrated training images. First, we compute the spatially-varying PSF of an optical lens using ray tracing [30], [31], and train a multilayer perceptron (MLP) to represent it. Once trained, the network can efficiently estimate the PSF for different object positions and focus distances.…”

Automatic Lens Design based on Differentiable Ray-tracing

“…However, this paraxial approximation is inaccurate and cannot optimize the refractive lens, limiting the designed optical systems to small fields of view and low aberration performance. Differentiable ray tracing [32,33,16,13] is another widely used approach for the simulation of refractive lenses and can be extended to simulate diffractive surfaces by approximating them as local gratings [25]. Ray tracing for diffractive surfaces has been employed in both commercial optical design software, such as Zemax [22], and recent research works [23,34,24].…”

“…End-to-end optical design [1,2,3,4,5,6] has demonstrated remarkable potential in scientific imaging [7,8,9,10,2,5,4,11], computer vision [3,12,13,14,15], and advanced optical system design [16,17], surpassing classical lens design approaches. An end-to-end lens system comprises an optical encoder, such as a refractive lens, the diffractive optical element (DOE), and/or metasurface [11,18], which captures information from the real world, and a neural network decoder that reconstructs the final output, which can be an image or other visual representation.…”

Embedded Processing for Extended Depth of Field Imaging Systems: From Infinite Impulse Response Wiener Filter to Learned Deconvolution

“…This means that digital processing parameters depend on optical system parameters, but it also means that optical system parameters depend on digital processing. Among the different digital processing techniques, neural networks can be used [3][4][5][6][7][8][9]. However, some applications (e.g., advanced driver assistance systems) require real-time digital processing, with a very limited memory storage ability.…”