Hybrid diffractive optics design via hardware-in-the-loop methodology for achromatic extended-depth-of-field imaging

Abstract

End-to-end optimization of diffractive optical elements (DOEs) profile through a digital differentiable model combined with computational imaging have gained an increasing attention in emerging applications due to the compactness of resultant physical setups. Despite recent works have shown the potential of this methodology to design optics, its performance in physical setups is still limited and affected by manufacturing artefacts of DOE, mismatch between simulated and resultant experimental point spread functions, and calibration errors. Additionally, the computational burden of the digital differentiable model to effectively design the DOE is increasing, thus limiting the size of the DOE that can be designed. To overcome the above mentioned limitations, a co-design of hybrid optics and image reconstruction algorithm is produced following the end-to-end hardware-in-the-loop strategy, using for optimization a convolutional neural network equipped with quantitative and qualitative loss functions. The optics of the imaging system consists on the phase-only spatial light modulator (SLM) as DOE and refractive lens. SLM phase-pattern is optimized by applying the Hardware-in-the-loop technique, which helps to eliminate the mismatch between numerical modelling and physical reality of image formation as light propagation is not numerically modelled but is physically done. Comparison with compound multi-lens optics of a last generation smartphone and a mirrorless commercial cameras show that the proposed system is advanced in all-in-focus sharp imaging for a depth range 0.4-1.9 m.

Miniature color camera via flat hybrid meta-optics

“…This contradiction is because the hardware is a “black box” of an unknown mathematical model and, as a result, is nondifferentiable. In addition, the efficiency of the CMA-ES method for black-box optimization of Image Signal Processor (ISP) is demonstrated by tuning a multi-objective highly nonlinear optimization problem in ( 3 ).…”

“…Thus, the miniaturization of cameras, while maintaining high image quality, has become a major driving force in optics and photonics research. A solution to this conundrum is computational imaging, where a digital backend augments the deficiencies of the optical components and improves the image quality, which has thus become a multidisciplinary research field at the intersection of optics, mathematics, and digital image processing ( 3 , 4 ).…”

“…In this work, we circumvent all these challenges by using a hybrid refractive/ meta-optics system in tandem with a computational backend, following a recently proposed design methodology ( 3 ). The desired phase profile is first optimized by a hardware-in-the-loop (HIL) strategy, where the DOE, implemented by a spatial light modulator (SLM), is configured and updated.…”

Curriculum Learning for ab initio Deep Learned Refractive Optics

“…This paradigm has been applied successfully to the design of single-element optical systems composed of a single diffractive optical element (DOE) or metasurface [23,12,10,1,7,29,15]. It has also been applied to the design of hybrid systems composed of an idealized thin lens combined with a DOE as an encoding element [3,25,27,18,11,22,20]. In the latter setting, the thin lens is used as an approximate representation of a pre-existing compound lens, while the DOE is designed encode additional information for specific imaging tasks such as high dynamic range imaging [25], hyperspectral imaging [12,10,1], sensor superresolution [23,27], extended depth of field [26], or cloaking of occluders [22].…”

Angular extended depth-of-focus diffractive lens based on Fourier series

“…[1][2][3] Several types of optical elements have been proposed to enlarge the depth of focus of lenses as apodizers, computer generated holograms, or diffractive optical elements (DOEs). [4][5][6] With in the last group, it is quite common to use amplitude or phase DOEs with radial or angular distribution. Some examples of radial DOEs are axicons, axilens, and optical elements with a certain number of rings.…”