Single-Shot Autofocusing of Microscopy Images Using Deep Learning

Abstract

Autofocusing is a critical step for high-quality microscopic imaging of specimens, especially for measurements that extend over time covering large fields-of-view. Autofocusing is generally practiced using two main approaches. Hardware-based optical autofocusing methods rely on additional distance sensors that are integrated with a microscopy system. Algorithmic autofocusing methods, on the other hand, regularly require axial scanning through the sample volume, leading to longer imaging times, which might also introduce phototoxicity and photobleaching on the sample. Here, we demonstrate a deep learning-based offline autofocusing method, termed Deep-R, that is trained to rapidly and blindly autofocus a single-shot microscopy image of a specimen that is acquired at an arbitrary out-of-focus plane. We illustrate the efficacy of Deep-R using various tissue sections that were imaged using fluorescence and brightfield microscopy modalities and demonstrate snapshot autofocusing under different scenarios, such as a uniform axial defocus as well as a sample tilt within the field-of-view. Our results reveal that Deep-R is significantly faster when compared with standard online algorithmic autofocusing methods. This deep learning-based blind autofocusing framework opens up new opportunities for rapid microscopic imaging of large sample areas, also reducing the photon dose on the sample. Focusing criterionAverage time (sec/mm 2 ) Standard deviation (sec/mm 2 ) Vollath4 39 42.91 3.16 Vollath5 39 39.57 3.16 Standard deviation 37.22 3.07 Normalized variance 10 36.50 0.36 Deep-R (CPU) 20.04 0.23 Deep-R (GPU) 2.98 0.08 Table. 1. Comparison of Deep-R computation time per 1 mm 2 of sample FOV (captured using a 20×/0.75NA objective lens) compared against other state-of-the-art autofocusing methods.

Label-free imaging of collagen fibers in tissue slices using phase imaging with computational specificity

“…The PSNR and SSIM values for the PICS inference were 31.7 and 0.75, respectively. The SSIM value from our model is within the acceptable level of performance according to previous reports [64]. One of the key features of both the original PLM image and PICS inference is that the glandular regions disappear, since glands are regions rich in epithelial cells that do not exhibit birefringence.…”

Single-Shot Diffraction Autofocusing: Distance Prediction via an Untrained Physics-Enhanced Network

“…Diffraction is a common optical phenomenon, and predicting the diffraction distance is significant for holographic reconstruction and imaging [20], [21], [24]- [26]. According to the angular spectrum theory, the diffraction process can be simulated accurately and calculated quickly in mathematics.…”

“…However, traditional autofocusing criteria, such as Contrast, L1-norm Gradient, Laplacian of Gaussian, Tenenbaum Gradient and Variance [22], [23], are all susceptible to misjudgment due to the noise and complex samples. The deep learningbased autofocusing methods, whether predicting reconstruction distances [24], [25], or generating focused images directly [26], are still suffering from the above drawbacks associated with supervised learning. In fact, the prediction of reconstruction distance is equal to that of diffraction distance in monochromatic coherent light.…”

Robust autofocusing for scanning electron microscopy based on a dual deep learning network

“…Moreover, recently, deep learning-based autofocus systems that have shown powerful performance in image processing are being actively studied 8 – 10 . Luo, et al 11 and Na, et al 12 respectively proposed an autofocus method for optical microscopy and SEM that receives a defocused image as an input and outputs a virtual focused image based on deep learning. Li, et al 13 developed a deep learning model that estimates the appropriate objective lens position by receiving two defocused images for light-sheet fluorescence microscopy.…”

“…In addition, SEM is based on electron beams instead of light, making it much more vulnerable to noise than optical microscopes due to factors such as electron absorption properties and charge-damaged artifacts 23 , 24 . Furthermore, deep learning-based autofocus research is also difficult to be widely applied to SEM, because most of the studies have been applied to optical microscopes based on numerical image quality metric, which have the aforementioned limitations 11 , 13 . Even when applied to SEM, there are limitations in the range of available WD and magnification 12 , 14 .…”