Applications, Promises, and Pitfalls of Deep Learning for Fluorescence Image Reconstruction

Abstract

Deep Learning is a recent and important addition to the computational toolbox available for image reconstruction in fluorescence microscopy. We review state-of-the-art applications such as image restoration, super-resolution, and light-field imaging, and discuss how the latest Deep Learning research can be applied to other image reconstruction tasks such as structured illumination, spectral deconvolution, and sample stabilisation. Despite its successes, Deep Learning also poses significant challenges, has often misunderstood capabilities, and overlooked limits. We will address key questions, such as: What are the challenges in obtaining training data? Can we discover structures not present in the training data? And, what is the danger of inferring unsubstantiated image details?

Topaz-Denoise: general deep denoising models for cryoEM and cryoET

“…Models generated using Topaz-Denoise, including the provided general models, may be susceptible to the hallucination problem in neural networks 30 . Details in denoised micrographs or tomograms may exhibit imprints of encodings from the datasets used for training.…”

Deep optical imaging within complex scattering media

“…The formation of an ideal image using MS waves is an underdetermined problem, and advances in computational tools, such as convolutional neural networks, may help in finding a solution by taking advantage of prior knowledge from mesoscopic physics. Deep learning based on neural networks has been actively used in recent years for image denoising, spatial and spectral deconvolution, super-resolution imaging, scattering noise reduction, image registration and tomographic reconstruction [142][143][144] . In the context of MS waves, neural networks have been used to train the input-output response for scattering media and multimode optical fibres 145,146 and to reconstruct refractive index maps for spatially confined objects from the MS waves of the objects themselves 147,148 .…”

A Review of Deep Learning Approaches for Inverse Scattering Problems (Invited Review)

“…Compared with the learning approach, the classical objective-function approach relies exclusively on first principles and inherently produces outputs that are consistent with their inputs [35]. In addition, there is no concern about the generalization since the classical objective-function approach works for any valid measurement.…”