Reinforcing neuron extraction and spike inference in calcium imaging using deep self-supervised denoising

Abstract

Calcium imaging is inherently susceptible to detection noise especially when imaging with high frame rate or under low excitation dosage. We developed DeepCAD, a selfsupervised learning method for spatiotemporal enhancement of calcium imaging without requiring any high signal-to-noise ratio (SNR) observations. Using this method, detection noise can be effectively suppressed and the imaging SNR can be improved more than tenfold, which massively improves the accuracy of neuron extraction and spike inference and facilitate the functional analysis of neural circuits.Calcium imaging enables parallel recordings of large neuronal ensembles in living animals [1][2][3][4] and offers a new possibility for deciphering information propagation, integration, and computation in neural circuits 5 . To obtain accurate neuron extraction and spike inference for downstream neuroscience analysis, high-SNR calcium imaging is desired. However, due to the paucity of fluorescence photons caused by low peak accumulations and fast dynamics of in vivo calcium transients 6,7 , calcium imaging is easy to be contaminated by detection noise (i.e. photon shot noise and electronic noise), especially in functional imaging where high temporal resolution is particularly important for analyzing neural activities 8 .To capture sufficient fluorescence photons for high-SNR calcium imaging, the most direct way is to use high excitation dosage, but concurrent photobleaching, phototoxicity 9,10 , and tissue heating 11 are detrimental for sample health and photosensitive biological processes, which limits the maximal excitation power for long-term in vivo imaging 12 . More effective .

Real-time denoising of fluorescence time-lapse imaging enables high-sensitivity observations of biological dynamics beyond the shot-noise limit

“…Moreover, we found that the combination of model simplification and data augmentation eliminates overfitting (Supplementary Fig. 6), which was an inherent problem of self-supervised training and required human inspections for model selection previously 32 . We compared DeepCAD-RT with DeepInterpolation, another recently developed denoising method leveraging inter-frame correlations 31 .…”

“…To fully exploit spatiotemporal correlations in fluorescence imaging data, all operations inside the network were implemented in 3D, including convolutions, max-poolings, and interpolations (Supplementary Figure 14). Compared to our previous architecture 32 , the number of feature maps in each convolutional layer was reduced by 4-fold and the total number of trainable parameters was reduced by 16-fold (1,020,337 compared with 16,315,585)， which massively improved the training and inference speed and reduced the memory consumption. For pre-processing, each input stack was subtracted by the average of the whole stack to handle the intensity variation across different samples and imaging platforms.…”

“…Comprehensive efforts have been invested to increase the photon budget of fluorescence microscopy, from designing high-performance fluorophores [23][24][25] , to upgrading the excitation and detection physics 19,[26][27][28] , and to developing data-driven denoising algorithms 22,[29][30][31] . We previously developed DeepCAD, a deep self-supervised denoising method for calcium imaging data, which effectively suppresses the detection noise and improves imaging SNR more than 10-fold without requiring any high-SNR observations 32 . A single low-SNR calcium imaging sequence can be directly used as the training data to train a denoising convolutional neural network.…”

Deep learning based domain adaptation for mitochondria segmentation on EM volumes

“…In the computer vision literature, related to natural images, the usefulness of pre-training has been widely explored for several tasks. Namely, the coloring of a grayscale image [50,51,52], the restoration of a distorted or deteriorated image [53,54,55,56], the prediction of the transformation performed in an image [57] or even, the re-ordering of pieces or frames of images [58,59] and videos [60]. However, there is hardly any work applying this methodology to microscopy images.…”

Imaging in focus: An introduction to denoising bioimages in the era of deep learning

“…Beyond simply denoising 2D images, many of the implementations described here have the capabilities of working on 3D dataset or even concomitant denoising of multiple channels, (see Table 1 for details). DeepCAD, a recent denoising implementation based on 3D U-Net ( Çiçek et al, 2016 ), efficiently improves the SNR of time-course calcium imaging ( Li et al, 2021 ). Here, using additional information from the context of the pixels in any relevant dimensions (3D, time or other channels for instance) to denoise often greatly improves denoising performance but at the expense of longer training times and the need for larger training datasets.…”