Extended depth-of-field in holographic imaging using deep-learning-based autofocusing and phase recovery

Abstract

Holography encodes the three dimensional (3D) information of a sample in the form of an intensity-only recording. However, to decode the original sample image from its hologram(s), auto-focusing and phase-recovery are needed, which are in general cumbersome and time-consuming to digitally perform. Here we demonstrate a convolutional neural network (CNN) based approach that simultaneously performs auto-focusing and phase-recovery to significantly extend the depth-of-field (DOF) in holographic image reconstruction. For this, a CNN is trained by using pairs of randomly de-focused back-propagated holograms and their corresponding in-focus phase-recovered images. After this training phase, the CNN takes a single back-propagated hologram of a 3D sample as input to rapidly achieve phase-recovery and reconstruct an in focus image of the sample over a significantly extended DOF. This deep learning based DOF extension method is non-iterative, and significantly improves the algorithm timecomplexity of holographic image reconstruction from O(nm) to O(1), where n refers to the number of individual object points or particles within the sample volume, and m represents the focusing search space within which each object point or particle needs to be individually focused. These results highlight some of the unique opportunities created by data-enabled statistical image reconstruction methods powered by machine learning, and we believe that the presented approach can be broadly applicable to computationally extend the DOF of other imaging modalities.

A neural lens for super-resolution biological imaging

“…Examples for such an application include, a 4x upscaling on photographic images [9], optical microscopy (improving the resolution from 40x to 100x) [10], dental imaging [11], phase imaging [12], fluorescence microscopy [13], magnetic resonance imaging [14], SEM imaging [15,16], positronemission tomography [17], stochastic optical reconstruction microscopy [18], and ultrasound imaging [19]. NNs are also well-suited to the classification of objects in images, and accordingly the classification of biological, pollution and colloidal particles from images and scattering patterns has also been demonstration [20][21][22][23][24].…”

Learning to synthesize: robust phase retrieval at low photon counts

“…Here, as in earlier works on direct phase retrieval [37][38][39][40][41][42][43], and due to the nonlinearity of the forward model, we adopt the End-to-End and Approximant methods. These we denote as End-to-End:f = DNN(g); and (8)…”

“…Recently, deep learning regression has been investigated for application to nonlinear inverse problems, in particular phase retrieval: direct [37][38][39], holographic [40,41], and ptychographic [42,43]. The idea, described briefly in Section 2.B, is to train a deep neural network (DNN) in supervised mode from examples of phase objects and their intensity images so that, after training, given an intensity image as input, the DNN outputs an estimate of the phase object.…”

Deep learning for the monitoring and process control of femtosecond laser machining

“…CNNs have been widely used in areas such as medical diagnostics [22], language translation [23], pollution detection [24] and the development of AI opponents in computer games [25]. In relation to photonics, neural networks have enabled improvements in optical microscopy [26] and Ptychography [27], light scattering control through opaque media [28] and object classification through scattering media [29,30], as well as for reconstructing ultrashort pulses, phase retrieval and holography [31,32].…”