Large depth-of-field ultra-compact microscope by progressive optimization and deep learning

Abstract

The optical microscope is customarily an instrument of substantial size and expense but limited performance. Here we report an integrated microscope that achieves the optical performance beyond a commercial microscope with a 5× objective but only at 0.15 cm3 and 0.5 gram, whose size is five orders of magnitude smaller than that of a conventional microscope. To achieve this, a progressive optimization pipeline is proposed which systematically optimizes both aspherical lenses and diffractive optical elements with over 30 times memory reduction compared to the end-to-end optimization. By designing a simulation-supervision deep neural network for spatially-varying deconvolution during optical design, we accomplish over 10-times improvement in the depth of field compared to traditional microscopes with great generalization in a wide variety of samples. To show the unique advantages, the integrated microscope is equipped in a cell phone without any accessories for the application of portable diagnostics. We believe our method provides a new framework for the design of miniaturized high-performance imaging systems by integrating aspherical optics, computational optics, and deep learning.

Point spread function engineering based on wavefront phase modulation

“…However, the actual optical system does not conform well to shift-invariance. And it is necessary to consider the MTFs/PSFs of different FOVs 68,85,86 , especially for systems with large FOV, which is often accomplished using optical design software. Finally, in some special scenarios where the scalar approximation does not hold, the optical model of wavefront coding will no longer be valid.…”

“…However, this method ignores aberrations and is therefore only suitable for optical systems with a small aberration such as microscopes. For imaging systems with non-negligible aberrations, PSFs/OTFs needs to be calculated based on optical design software 68,80,85,86,[99][100][101] . This way can take actual aberrations into account and achieve better imaging quality, but requires specific design and lacks flexibility.…”

“…The invariance of the PSF to aberrations can be characterized by Strehl ratio 89 , Fisher information 89,97 , mean square error 26,90,96,102 , and variance 103 . The consistency of MTF/OTF to aberrations can be characterized by Fisher information 64,72,73,85 , mean square error 74,80 , Hilbert space angle 68,69,72,76 , and standard deviation 104 . Finally, the parameters of the phase mask need to be optimized based on the consistency of PSF and OTF to aberrations.…”

“…However, all these deconvolution algorithms assume that the blur kernel is shift-invariant, which is obviously not in line with the shift characteristics of real optical systems and will lead to artifacts in the reconstructed image 113 . The shiftvariant deconvolution algorithm 85,114,115 models the shift-variant PSF as a weighted superposition of multiple shiftinvariant kernels, where the weights change with the FOV, which allows a large number of PSFs to be represented with very little data. With the help of deep neural networks, it is possible to perform shift deconvolution that is fast, low-noise, and has good recovery of high-frequency information 85 .…”

“…The shiftvariant deconvolution algorithm 85,114,115 models the shift-variant PSF as a weighted superposition of multiple shiftinvariant kernels, where the weights change with the FOV, which allows a large number of PSFs to be represented with very little data. With the help of deep neural networks, it is possible to perform shift deconvolution that is fast, low-noise, and has good recovery of high-frequency information 85 . Shift-variant deconvolution algorithm can compensate for the shift-variance caused by aberrations, imperfect designs, manufacturing and installation errors, and improve the quality of image reconstruction.…”