Advancing Spectrally‐Resolved Single Molecule Localization Microscopy with Deep Learning

Abstract

Spectrally‐resolved single molecule localization microscopy (srSMLM) is a recent technique enriching single molecule localization microscopy with the simultaneous recording of spectra of the single emitters. srSMLM resolution is limited by the number of photons collected per emitters. Sharing a photon budget to record the localization and the spectroscopic information results in a loss of spatial and spectral resolution—or forces the sacrifice of one at the expense of the other. Here, srUnet—a deep‐learning Unet‐based image processing routine trained to increase the spectral and spatial signals to compensate for the resolution loss inherent in additionally recording the spectral component is reported. Both localization and spectral precision are improved by srUnet—particularly for the low‐emitting species. srUnet increases the fraction of localization whose signal can be both spatially and spectrally characterized. It preserves spectral shifts and the linearity of the dispersion of light. It strongly facilitates wavelength assignment in multicolor experiments. srUnet is a simple post‐processing add‐on boosting srSMLM performance close to conventional SMLM with the potential to turn srSMLM into the new standard for multicolor single molecule imaging.

Deep-learning-assisted spectroscopic single-molecule localization microscopy based on spectrum-to-spectrum denoising

“…Additionally, Spec2Spec can restore the spectral variations caused by the underlying fluorescence heterogeneity. 11,20 As shown in Fig. 4m and n, the minute spectral variations from the same type of dye molecules, which are drowned out by noise and are difficult to distinguish from each other in the raw data, can be clearly resolved in the denoised data.…”

“…The training network in such a supervised manner depends heavily on paired ground truth (GT) signals. 20 However, in the context of sSMLM imaging, it is challenging to obtain an unbiased spectral signal due to the fast dynamics and inherent spectral heterogeneity of stochastic single-molecule emission. 11,21…”

“…The training network in such a supervised manner depends heavily on paired ground truth (GT) signals. 20 However, in the context of sSMLM imaging, it is challenging to obtain an unbiased spectral signal due to the fast dynamics and inherent spectral heterogeneity of stochastic single-molecule emission. 11,21 In this paper, we propose the spectrum-to-spectrum (Spec2Spec) framework, a novel self-supervised deep-learningbased spectral denoising method to unbiasedly remove the noise from single-molecule emission spectra in sSMLM.…”

Spectral Phasor Applied to Spectrally‐Resolved Single Molecule Localization Microscopy

“…Spectra were then extracted for each localization using python routine. [15] The phasors were calculated using spectralPhasoR -a package for spectral Phasor representation of spectrally-resolved single molecule localisation microscopy data, available on git.unistra.fr.…”

“…In particular, scrutinizing the spectral dimension necessitates appropriate and dedicated tools. [13] Very recently, some hardware attempts [14] or software solutions based on deep learning [15] have been proposed to improve the spectral resolution. Exploring the spectral dimension more extensively would be intriguing for harnessing the potential of any dye that demonstrates both an environment-specific spectral shift and can be precisely localized at the singlemolecule level.…”