Applications, Promises, and Pitfalls of Deep Learning for Fluorescence Image Reconstruction

Abstract

Deep Learning is a recent and important addition to the computational toolbox available for image reconstruction in fluorescence microscopy. We review state-of-the-art applications such as image restoration, super-resolution, and light-field imaging, and discuss how the latest Deep Learning research can be applied to other image reconstruction tasks such as structured illumination, spectral deconvolution, and sample stabilisation. Despite its successes, Deep Learning also poses significant challenges, has often misunderstood capabilities, and overlooked limits. We will address key questions, such as: What are the challenges in obtaining training data? Can we discover structures not present in the training data? And, what is the danger of inferring unsubstantiated image details?

Batch Equalization with a Generative Adversarial Network

“…The ability to generate hypothetical datasets opens up a new possibility for how to use deep learning to understand a dataset. Standard applications of deep learning to microscope image preprocessing are focused on exposing unseen details such as super-resolution (Wang et al, 2019) or image reconstruction (Belthangady and Royer, 2019). However, deep learning may not only be useful in analysis, but it could also be useful in data exploration based on transformations between conditions.…”

Widefield light sheet microscopy using an Airy beam combined with deep-learning super-resolution

“…In LSM, deep learning has been applied previously in one instance, however, the training was performed using artificially simulated low-resolution images from high-resolution data [27]. In microscopy, deep-learning has been typically used to surpass the diffraction limit to achieve super-resolution microscopy [28]. However, in a broader context of image processing, deep-learning has been applied to a class of problems termed single-image super-resolution [29], the goal of which is to enhance resolution in poor sampling regimes.…”

“…The use of deep-learning in microscopy has to be taken with care [28]. There are a variety of network models that can generate false features based on the features observed in the training data.…”

Deep learning enables structured illumination microscopy with low light levels and enhanced speed

“…For microscopy, deep learning has demonstrated impressive capabilities in cell segmentation/tracking, morphology analysis, denoising, single molecule detection/tracking, and super-resolution imaging. [7][8][9] Use of deep learning for content-aware image restoration (CARE) has shown great promise in de-noising, enhancing signal to noise ratio, and isotropic imaging 10 . However, the potential of deep learning in SIM has not been explored.…”