Deep learning‐based color holographic microscopy

Liu, Tairan; Wei, Zhensong; Rivenson, Yair; Haan, Kevin de; Zhang, Hongjie; Wu, Yichen; Ozcan, Aydogan

doi:10.1002/jbio.201900107

Cited by 43 publications

(32 citation statements)

References 66 publications

(100 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Convolutional neural network (CNN) and deep learning approaches have been proposed for several optical applications. Examples include virtual staining of non-stained samples [33], increasing spatial resolution in a large field of view in optical microscopy [34,35], color holographic microscopy with CNN [36], autofocusing and enhancing the depth-of-filed in inline holography [37], lens-less computational imaging by deep learning [38], single-cell-based reconstruction distance estimation by a regression CNN model [39], super-resolution fringe patterns by deep learning holography [40], virtual refocusing in fluorescence microscopy to map 2D images to a 3D surface [41], and several other studies [42][43][44]. Deep-learning based phase recovery by residual CNN model was also suggested [45], but the application is limited because the reference noise-free phase images for deep-learning model are generated by the multi-height phase retrieval approach (8 holograms are recorded at different sample-to-sensor distances).…”

Section: Proposed Deep Learning Model For Phase Recoverymentioning

confidence: 99%

Noise-free quantitative phase imaging in Gabor holography with conditional generative adversarial network

Moon¹,

Jaferzadeh²,

Kim³

et al. 2020

Opt. Express

View full text Add to dashboard Cite

This paper shows that deep learning can eliminate the superimposed twin-image noise in phase images of Gabor holographic setup. This is achieved by the conditional generative adversarial model (C-GAN), trained by input-output pairs of noisy phase images obtained from synthetic Gabor holography and the corresponding quantitative noise-free contrast-phase image obtained by the off-axis digital holography. To train the model, Gabor holograms are generated from digital off-axis holograms with spatial shifting of the real image and twin image in the frequency domain and then adding them with the DC term in the spatial domain. Finally, the digital propagation of the Gabor hologram with Fresnel approximation generates a superimposed phase image for the C-GAN model input. Two models were trained: a human red blood cell model and an elliptical cancer cell model. Following the training, several quantitative analyses were conducted on the biochemical properties and similarity between actual noise-free phase images and the model output. Surprisingly, it is discovered that our model can recover other elliptical cell lines that were not observed during the training. Additionally, some misalignments can also be compensated with the trained model. Particularly, if the reconstruction distance is somewhat incorrect, this model can still retrieve in-focus images.

show abstract

Section: Proposed Deep Learning Model For Phase Recoverymentioning

confidence: 99%

Noise-free quantitative phase imaging in Gabor holography with conditional generative adversarial network

Moon¹,

Jaferzadeh²,

Kim³

et al. 2020

Opt. Express

View full text Add to dashboard Cite

show abstract

“…Recent works transformed auto-fluorescence images [ 24 ] or hyperspectral images [ 25 ] into histopathologically stained H&E images using CGANs. A similar approach was performed for translating quantitative phase imaging into three different stains, namely H&E stain, Jone’s stain, and Masson’s trichrome stain [ 26 ]. CGANs were also employed to increase the spatial resolution [ 27 , 28 ] and remove speckle noise from optical microscopic images.…”

Section: Introductionmentioning

confidence: 99%

Computational tissue staining of non-linear multimodal imaging using supervised and unsupervised deep learning

Pradhan

Meyer

Stallmach

et al. 2021

Biomed. Opt. Express

View full text Add to dashboard Cite

Hematoxylin and Eosin (H&E) staining is the ’gold-standard’ method in histopathology. However, standard H&E staining of high-quality tissue sections requires long sample preparation times including sample embedding, which restricts its application for ’real-time’ disease diagnosis. Due to this reason, a label-free alternative technique like non-linear multimodal (NLM) imaging, which is the combination of three non-linear optical modalities including coherent anti-Stokes Raman scattering, two-photon excitation fluorescence and second-harmonic generation, is proposed in this work. To correlate the information of the NLM images with H&E images, this work proposes computational staining of NLM images using deep learning models in a supervised and an unsupervised approach. In the supervised and the unsupervised approach, conditional generative adversarial networks (CGANs) and cycle conditional generative adversarial networks (cycle CGANs) are used, respectively. Both CGAN and cycle CGAN models generate pseudo H&E images, which are quantitatively analyzed based on mean squared error, structure similarity index and color shading similarity index. The mean of the three metrics calculated for the computationally generated H&E images indicate significant performance. Thus, utilizing CGAN and cycle CGAN models for computational staining is beneficial for diagnostic applications without performing a laboratory-based staining procedure. To the author’s best knowledge, it is the first time that NLM images are computationally stained to H&E images using GANs in an unsupervised manner.

show abstract

“…Fang et al have developed an on-chip lensless flow cytometer, which uses the holographic imaging principle and the broad FOV of the S-shaped pipe to achieve a cell counting error of less than 2% [32]. Since the experimental device of holographic imaging is simple, and the resolution can be improved through phase recovery, it has been widely studied and applied in recent years [33][34][35].…”

Section: Introductionmentioning

confidence: 99%

High-Precision Lensless Microscope on a Chip Based on In-Line Holographic Imaging

Huang

et al. 2021

Sensors

View full text Add to dashboard Cite

The lensless on-chip microscope is an emerging technology in the recent decade that can realize the imaging and analysis of biological samples with a wide field-of-view without huge optical devices and any lenses. Because of its small size, low cost, and being easy to hold and operate, it can be used as an alternative tool for large microscopes in resource-poor or remote areas, which is of great significance for the diagnosis, treatment, and prevention of diseases. To improve the low-resolution characteristics of the existing lensless shadow imaging systems and to meet the high-resolution needs of point-of-care testing, here, we propose a high-precision on-chip microscope based on in-line holographic technology. We demonstrated the ability of the iterative phase recovery algorithm to recover sample information and evaluated it with image quality evaluation algorithms with or without reference. The results showed that the resolution of the holographic image after iterative phase recovery is 1.41 times that of traditional shadow imaging. Moreover, we used machine learning tools to identify and count the mixed samples of mouse ascites tumor cells and micro-particles that were iterative phase recovered. The results showed that the on-chip cell counter had high-precision counting characteristics as compared with manual counting of the microscope reference image. Therefore, the proposed high-precision lensless microscope on a chip based on in-line holographic imaging provides one promising solution for future point-of-care testing (POCT).

show abstract

Deep learning‐based color holographic microscopy

Cited by 43 publications

References 66 publications

Noise-free quantitative phase imaging in Gabor holography with conditional generative adversarial network

Noise-free quantitative phase imaging in Gabor holography with conditional generative adversarial network

Computational tissue staining of non-linear multimodal imaging using supervised and unsupervised deep learning

High-Precision Lensless Microscope on a Chip Based on In-Line Holographic Imaging

Contact Info

Product

Resources

About