Deep learning-based super-resolution in coherent imaging systems

Liu, Tairan; Haan, Kevin de; Rivenson, Yair; Wei, Zhensong; Zeng, Xin; Zhang, Hongjie; Ozcan, Aydogan

doi:10.1038/s41598-019-40554-1

Cited by 112 publications

(84 citation statements)

References 44 publications

(51 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep-Z framework enables digital refocusing of out-of-focus 3D features in a wide-field fluorescence microscope image to user-defined surfaces. The same concept can also be used to Deep learning has also been recently demonstrated to be very effective in performing deconvolution to boost the lateral [32][33][34][35][36][37] and the axial 38,39 resolution in microscopy images. Deep-Z network reported here is unique as it selectively deconvolves the spatial features that come into focus through the digital refocusing process (see e.g.…”

Section: Cross-modality Digital Refocusing Of Fluorescence Images: Dementioning

confidence: 99%

Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning

et al. 2019

Self Cite

View full text Add to dashboard Cite

Three-dimensional (3D) fluorescence microscopy in general requires axial scanning to capture images of a sample at different planes. Here we demonstrate that a deep convolutional neural network can be trained to virtually refocus a 2D fluorescence image onto user-defined 3D surfaces within the sample volume. With this data-driven computational microscopy framework, we imaged the neuron activity of a Caenorhabditis elegans worm in 3D using a time-sequence of fluorescence images acquired at a single focal plane, digitally increasing the depth-of-field of the microscope by 20-fold without any axial scanning, additional hardware, or a trade-off of imaging resolution or speed. Furthermore, we demonstrate that this learning-based approach can correct for sample drift, tilt, and other image aberrations, all digitally performed after the acquisition of a single fluorescence image. This unique framework also cross-connects different imaging modalities to each other, enabling 3D refocusing of a single wide-field fluorescence image to match confocal microscopy images acquired at different sample planes. This deep learning-based 3D image refocusing method might be transformative for imaging and tracking of 3D biological samples, especially over extended periods of time, mitigating photo-toxicity, sample drift, aberration and defocusing related challenges associated with standard 3D fluorescence microscopy techniques. Text:Three-dimensional (3D) fluorescence microscopic imaging is essential for biomedical and physical sciences as well as engineering, covering various applications 1-7 . Despite its broad

show abstract

Section: Cross-modality Digital Refocusing Of Fluorescence Images: Dementioning

confidence: 99%

Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…Since increasing the pre-training data improved the accuracy of the denoised images, adding more pre-training data will improve the accuracy. For further enhancement of the accuracy, generative adversarial networks (GANs) 35,36 , which can enhance the performance by using a pair of two networks, are strong candidates.…”

Section: Discussionmentioning

confidence: 99%

Improvement of nerve imaging speed with coherent anti-Stokes Raman scattering rigid endoscope using deep-learning noise reduction

Yamato

Niioka

Miyake

et al. 2020

Sci Rep

View full text Add to dashboard Cite

A coherent anti-Stokes Raman scattering (CARS) rigid endoscope was developed to visualize peripheral nerves without labeling for nerve-sparing endoscopic surgery. The developed CARS endoscope had a problem with low imaging speed, i.e. low imaging rate. In this study, we demonstrate that noise reduction with deep learning boosts the nerve imaging speed with CARS endoscopy. We employ fine-tuning and ensemble learning and compare deep learning models with three different architectures. In the fine-tuning strategy, deep learning models are pre-trained with CARS microscopy nerve images and retrained with CARS endoscopy nerve images to compensate for the small dataset of CARS endoscopy images. We propose using the equivalent imaging rate (EIR) as a new evaluation metric for quantitatively and directly assessing the imaging rate improvement by deep learning models. The highest EIR of the deep learning model was 7.0 images/min, which was 5 times higher than that of the raw endoscopic image of 1.4 images/min. We believe that the improvement of the nerve imaging speed will open up the possibility of reducing postoperative dysfunction by intraoperative nerve identification.

show abstract

“…Deep learning has also been demonstrated to efficiently enhance the resolution of lensless microscopy systems using significantly fewer images compared to these earlier pixel super-resolution methods, which further increases the throughput of these lensless systems and relaxes some of their hardware design constraints [see Fig. 8(c)] [59].…”

Section: P H Y S I C a L M O D E L-b A S E D I M A G E R E C O mentioning

confidence: 99%

Deep-Learning-Based Image Reconstruction and Enhancement in Optical Microscopy

et al. 2020

Self Cite

View full text Add to dashboard Cite

This article provides an overview of efforts to advance the field of computational microscopy and optical sensing systems for microscopy using deep neural networks. First, the work overviews the basics of inverse problems in optical microscopy and then outlines how deep learning can be a framework for solving these problems, typically through supervised methods. Then, there is a discussion of use of deep learning to try to obtain single-image super resolution and image enhancement in these data sets.

show abstract

Deep learning-based super-resolution in coherent imaging systems

Cited by 112 publications

References 44 publications

Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning

Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning

Improvement of nerve imaging speed with coherent anti-Stokes Raman scattering rigid endoscope using deep-learning noise reduction

Deep-Learning-Based Image Reconstruction and Enhancement in Optical Microscopy

Contact Info

Product

Resources

About