O-Net: A Fast and Precise Deep-Learning Architecture for Computational Super-Resolved Phase-Modulated Optical Microscopy

Abstract

We present a fast and precise deep-learning architecture, which we term O-Net, for obtaining super-resolved images from conventional phase-modulated optical microscopical techniques, such as phase-contrast microscopy and differential interference contrast microscopy. O-Net represents a novel deep convolutional neural network that can be trained on both simulated and experimental data, the latter of which is being demonstrated in the present context. The present study demonstrates the ability of the proposed method to achieve super-resolved images even under poor signal-to-noise ratios and does not require prior information on the point spread function or optical character of the system. Moreover, unlike previous state-of-the-art deep neural networks (such as U-Nets), the O-Net architecture seemingly demonstrates an immunity to network hallucination, a commonly cited issue caused by network overfitting when U-Nets are employed. Models derived from the proposed O-Net architecture are validated through empirical comparison with a similar sample imaged via scanning electron microscopy (SEM) and are found to generate ultra-resolved images which came close to that of the actual SEM micrograph.

Θ-Net: Achieving Enhanced Phase-Modulated Optical Nanoscopyin silicothrough a computational‘string of beads’architecture

“…Here, fn+1 refers to the generated 'PSF' of the image when (fn ⊛ g) is mapped under jn. Like O-Net [28], the -Net models thus attempt to learn the mapping function jn across n nodes (for which -Net is defined), and (upon sufficiently learning this) deploys jn for image SR as follows: Mapping Phase…”

“…Nonetheless, a key aspect of our present study remainsi.e. to highlight the superiority of models adopting the -Net framework over their predecessors employing the O-Net framework (as described in [28]) for in silico label-free SR microscopyevidence for this being presented in Figures 4 & 6 respectively. As previously enunciated, -Net (in the present study) employs a triple-node architecture (each node being an O-Net) with a specialized transfer-learnt O-Net model for the last node (the said node being trained on both DIC & PCM datasets).…”

“…where bi is the label and 𝑏 ̂𝑖 is the probability of bi = 1 (derived from [44] & [45], and described in [28]). In contrast, for computing the generator (g) loss, both  and the mean absolute error (MAE) / l1 loss were used, the latter being computed as follows(from [28] and [46]):…”

“…At this juncture, it would be essential to highlight that the individual O-Net nodes within a -Net chain are founded on the Pix2Pix GAN architecture, as discussed in [28] & [31]. Similarly, Swish [32] & GELU [33] activation functions were also incorporated into the encoder & decoder blocks respectively (the mathematical definitions of these functions being included in the accompanying Supplement for the interested reader).…”

Small Training Dataset Convolutional Neural Networks for Application Specific Super-Resolution Microscopy

“…Besides, our demonstrated model can work with other super-resolution target generation methods like STED/STORM/PALM/SIM. While we evaluated the technique on super-resolution fluorescence microscopy, this approach shows promise for extension to other deep learning based image enhancements (e.g., image denoising networks [10, 43], image super-resolution [26, 44, 45, 46, 47], image segmentation networks [28], and other imaging modalities like X-ray [48, 49, 50] and MRI imaging [51]).…”