DeepRegularizer: Rapid Resolution Enhancement of Tomographic Imaging Using Deep Learning

Abstract

Optical diffraction tomography measures the three-dimensional refractive index map of a specimen and visualizes biochemical phenomena at the nanoscale in a non-destructive manner. One major drawback of optical diffraction tomography is poor axial resolution due to limited access to the three-dimensional optical transfer function. This missing cone problem has been addressed through regularization algorithms that use a priori information, such as non-negativity and sample smoothness. However, the iterative nature of these algorithms and their parameter dependency make real-time visualization impossible. In this article, we propose and experimentally demonstrate a deep neural network, which we term DeepRegularizer, that rapidly improves the resolution of a three-dimensional refractive index map. Trained with pairs of datasets (a raw refractive index tomogram and a resolution-enhanced refractive index tomogram via the iterative total variation algorithm), the three-dimensional U-net-based convolutional neural network learns a transformation between the two tomogram domains. The feasibility and generalizability of our network are demonstrated using bacterial cells and a human leukaemic cell line, and by validating the model across different samples. DeepRegularizer offers more than an order of magnitude faster regularization performance compared to the conventional iterative method. We envision that the proposed data-driven approach can bypass the high time complexity of various image reconstructions in other imaging modalities.

Roadmap on Digital Holography-Based Quantitative Phase Imaging

“…The IDT system has more flexibility in controlling the free-floating sample, and it is possible to extend the spatial frequency coverage along the axial direction. The continuous development and progressive solutions by the researchers in computational, experimental, and artificial intelligence algorithms has brought enhanced measurement accuracy to technique, increased certainty of analysis [112][113][114][115][131][132][133], as well as access to new multimodal techniques and functionalities [134][135][136][137][138][139][140].…”

“…At present, 3D QPI has enhanced its implementations by combining techniques, such as fluorescence or Raman imaging, into multimodal operations [135][136][137][138][139][140][141]152,153,155]. Recent developments in artificial intelligence algorithms and machine learning approaches are now the focus in 3D QPI systems, aimed at improving system architecture and measurement accuracy in a more effective way [112][113][114][115][131][132][133]176,177].…”

“…Revolutionary advances in AI have addressed several challenges of 3D QPI. Deep learning approaches have been used in image acquisition and reconstruction, including denoising and deblurring [112]. Reconstructed images have also been analyzed, with the aid of AI, for biological studies, including cell type classifications (Figure 8a) [113] and segmentation [114].…”

3D imaging lipidometry in single cell by in-flow holographic tomography

“…However, to ensure these advantages, the rolling angles are not known a priori, although they are requested to perform the tomographic reconstruction. Recently, we proposed a rolling angles recovery method to solve this issue, based on the identification of phase similarities 39 . The presence of intracellular LDs provides a great help in making more accurate the rolling angles recovery and then the 3D RI tomogram.…”

Rapid label-free identification of pathogenic bacteria species from a minute quantity exploiting three-dimensional quantitative phase imaging and artificial neural network

“…Among various biomedical studies, bacteria has been investigated with QPI during growth (Ahn et al, 2020;Mir et al, 2011), while optically controlled in the presence of eukaryotic cells (Kemper et al, 2013), and upon the treatments of antibotics (Oh et al, 2020). In recent years, machine learning has been introduced to QPI (Jo et al, 2018;Rivenson et al, 2019b), enabling diverse applications including virtual staining (Rivenson et al, 2019a), virtual molecular imaging (Jo et al, 2020;Kandel et al, 2020), improvement of image quality (Kamilov et al, 2015;Ryu et al, 2019;Ryu et al, 2021), and a variety cell type classification (Chen et al, 2016;Rubin et al, 2019;Siu et al, 2020;Yoon et al, 2017). One noteworthy study realized efficient screening for anthrax spores using a handheld twodimensional (2D) QPI microscope and artificial neural network (ANN) (Jo et al, 2017).…”