Comparing synthetic refocusing to deconvolution for the extraction of neuronal calcium transients from light fields

Howe, Carmel L.; Quicke, Peter; Song, Pingfan; Jadan, Herman Verinaz; Dragotti, Pier Luigi; Foust, Amanda J.

doi:10.1117/1.nph.9.4.041404

Cited by 4 publications

(4 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, they need huge training datasets, low background noise, non-scattering media, or transparent samples. In contrast, model-based approaches are more robust than learning methods and helpful in adverse conditions, as shown in works studying mammalian brain tissue [12], [13]. Thus, we propose fusing appealing features from model-based and learning-based methods.…”

Section: Previous Workmentioning

confidence: 99%

See 1 more Smart Citation

Physics-Based Deep Learning for Imaging Neuronal Activity via Two-Photon and Light Field Microscopy

Jadan

Howe

Song

et al. 2023

IEEE Trans. Comput. Imaging

Self Cite

View full text Add to dashboard Cite

Light Field Microscopy (LFM) is an imaging technique that offers the opportunity to study fast dynamics in biological systems due to its 3D imaging speed and is particularly attractive for functional neuroimaging. Traditional model-based approaches employed in microscopy for reconstructing 3D images from light-field data are affected by reconstruction artifacts and are computationally demanding. This work introduces a deep neural network for LFM to image neuronal activity under adverse conditions: limited training data, background noise, and scattering mammalian brain tissue. The architecture of the network is obtained by unfolding the ISTA algorithm and is based on the observation that neurons in the tissue are sparse. Our approach is also based on a novel modelling of the imaging system that uses a linear convolutional neural network to fit the physics of the acquisition process. We train the network in a semi-supervised manner based on an adversarial training framework. The small labelled dataset required for training is acquired from a single sample via two-photon microscopy, a point-scanning 3D imaging technique that achieves high spatial resolution and deep tissue penetration but at a lower speed than LFM. We introduce physics knowledge of the system in the design of the network architecture and during training to complete our semi-supervised approach. We experimentally show that in the proposed scenario, our method performs better than typical deep learning and model-based reconstruction strategies for imaging neuronal activity in mammalian brain tissue via LFM, considering reconstruction quality, generalization to functional imaging, and reconstruction speed.

show abstract

Section: Previous Workmentioning

confidence: 99%

“…Testing is performed on structural LF stacks or functional LF sequences, and we produce one volume per frame. Neuronal activity is extracted from these volumes, as illustrated in part (b) of our approach conditions [12], [13].…”

Section: Introductionmentioning

confidence: 99%

Physics-Based Deep Learning for Imaging Neuronal Activity via Two-Photon and Light Field Microscopy

Jadan

Howe

Song

et al. 2023

IEEE Trans. Comput. Imaging

Self Cite

View full text Add to dashboard Cite

show abstract

“…For instance, when studying neuronal activity in mammalian brain tissue, the sample is highly scattering, nontransparent and contains high background noise, which makes training artificial neural networks (ANN) challenging. On the other hand, model-based optimization approaches have shown to be more robust under these adverse experimental conditions [12], [13]. This work proposes a novel multimodal imaging approach leveraging the respective strengths of 2P microscopy and LFM.…”

Section: Introductionmentioning

confidence: 99%

Physics-based Deep Learning for Imaging Neuronal Activity via Two-photon and Light Field Microscopy

Jadan

Howe

Song

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Light Field Microscopy (LFM) is an imaging technique that offers the opportunity to study fast dynamics in biological systems due to its rapid 3D imaging rate. In particular, it is attractive to analyze neuronal activity in the brain. Unlike scanning-based imaging methods, LFM simultaneously encodes the spatial and angular information of light in a single snapshot. However, LFM is limited by a trade-off between spatial and angular resolution and is affected by scattering at deep layers in the brain tissue. In contrast, two-photon (2P) microscopy is a point-scanning 3D imaging technique that achieves higher spatial resolution, deeper tissue penetration, and reduced scattering effects. However, point-scanning acquisition limits the imaging speed in 2P microscopy and cannot be used to simultaneously monitor the activity of a large population of neurons. This work introduces a physics-driven deep neural network to image neuronal activity in scattering volume tissues using LFM. The architecture of the network is obtained by unfolding the ISTA algorithm and is based on the observation that the neurons in the tissue are sparse. The deep-network architecture is also based on a novel imaging system modeling that uses a linear convolutional neural network and fits the physics of the acquisition process. To achieve the high-quality reconstruction of neuronal activity in 3D brain tissues from temporal sequences of light field (LF) images, we train the network in a semi-supervised manner using generative adversarial networks (GANs). We use the TdTomato indicator to obtain static structural information of the tissue with the microscope operating in 2P scanning modality, representing the target reconstruction quality. We also use additional functional data in LF modality with GCaMP indicators to train the network. Our approach is tested under adverse conditions: limited training data, background noise, and scattering samples. We experimentally show that our method performs better than model-based reconstruction strategies and typical artificial neural networks for imaging neuronal activity in mammalian brain tissue, considering reconstruction quality, generalization to functional imaging, and reconstruction speed.

show abstract

“…(issue cover) provide a focused review on advances in computer-generated holography (CGH) for targeted neuronal modulation, which highlights novel algorithms and hardware implementations of advanced CGH techniques to shape the light intensity distribution in 3D for optically interrogating neuronal populations in conjugation with optogenetics. Howe et al 10 . assessed a popular computational imaging technique, known as light field microscopy, for the extraction of neuronal calcium transients in 3D from a single-shot measurement.…”

mentioning

confidence: 99%

Special Section Guest Editorial: Computational Approaches for Neuroimaging

2022

View full text Add to dashboard Cite

New advances in computational methods are revolutionizing our ability to collect, reconstruct, analyze, and interpret neuroimaging data. In turn, they provide unique new approaches to understand brain activities and functions at subcellular, cellular, circuit and network levels. This special section collates ten contributions that cover many recent computational technologies underlying these developments. These computational techniques address several challenges, including recording of neural activities, reconstruction of neural activities, inference of functional connectivity, and interpretation of the brain function at cellular level and circuit level. This special section of Neurophotonics Volume 9 Issue 4 consists of five review articles and five research papers. These contributions are representative of the broad range of optical technologies employed in neuroscience, such as one-photon or multi-photon imaging of functional indicators, e.g., calcium, voltage, and neurotransmitter, functional near-infrared spectroscopies to image hemodynamics, and optical intrinsic signal imaging. Beyond imaging, this special section also covers the rising field of photostimulation.A major focus of emerging computational approaches is towards fast and accurate data processing for functional neuroimaging data. Benisty et al. 1 provide a comprehensive review on data processing of functional optical microscopy for neuroscience, which surveys a broad range of techniques to handle massive spatiotemporal datasets from fluorescent microscopes in order to uncover neuronal activity related to behavior and stimuli, and local circuits in the brain. Cai et al. 2 provide a focused review on data analysis methods for mesoscale neural imaging in vivo, which is timely since mesoscale imaging at high resolution has become one of the main frontier in neuroimaging. Carrillo-Reid and Calderon 3 present a review on conceptual framework for neuronal ensemble identification and manipulation related to behavior using calcium imaging, which discusses computational approaches to infer neuronal ensembles from calcium imaging in behaving mice. Eastmond et al. 4 offer a comprehensive review on deep learning in functional near-infrared spectroscopy (fNIRS), which covers the many applications of deep learning in fNIRS for brain-computer interface, neuro-impairment diagnosis, and neuroscience discovery. Gao et al. 5 report a novel denoising autoencoder deep learning model to remove motion artifacts that plague fNIRS data in real world applications. White et al. 6 compare data processing methods for optical intrinsic signal imaging to analyze resting-state functional connectivity in mice by processing temporal and spatial autocorrelations. Dehkharghanian et al. 7 present a new semi-automated machine learning algorithm for analyzing bioluminescence images in order to measure adenosine triphosphate (ATP) indicators in neurons.Another emerging area is computational imaging, which seeks to synergistically combine novel optical designs and advanced computational ...

show abstract

Comparing synthetic refocusing to deconvolution for the extraction of neuronal calcium transients from light fields

Cited by 4 publications

References 50 publications

Physics-Based Deep Learning for Imaging Neuronal Activity via Two-Photon and Light Field Microscopy

Physics-Based Deep Learning for Imaging Neuronal Activity via Two-Photon and Light Field Microscopy

Physics-based Deep Learning for Imaging Neuronal Activity via Two-photon and Light Field Microscopy

Special Section Guest Editorial: Computational Approaches for Neuroimaging

Contact Info

Product

Resources

About