A practical guide to deep-learning light-field microscopy for 3D imaging of biological dynamics

Zhu, Lanxin; Yi, Chengqiang; Peng, Fei

doi:10.1016/j.xpro.2023.102078

Cited by 5 publications

(3 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, the literature presents several strategies to enhance SLIM's performance, such as background rejection by hardware 17 and computation 52 , multi-focus optics for extended DoF 15,53 , and sparsity-based resolution enhancement 43,54 . Furthermore, ongoing advancements in datadriven reconstruction algorithms, particularly physics-embedded deep learning models 19,20,45,55 , hold great promise for addressing the ill-posed inverse problems associated with limited spacebandwidth and compressive detection in SLIM. These developments are expected to significantly enhance SLIM's capabilities and broaden its utility across diverse imaging scenarios.…”

Section: Discussionmentioning

confidence: 99%

Kilohertz volumetric imaging of in-vivo dynamics using squeezed light field microscopy

Wang,

Zhao,

Wagenaar

et al. 2024

Preprint

View full text Add to dashboard Cite

Volumetric functional imaging of transient cellular signaling and motion dynamics poses a significant challenge to current microscopy techniques, primarily due to limitations in hardware bandwidth and the restricted photon budget within short exposure times. In response to this challenge, we present squeezed light field microscopy (SLIM), a computational imaging method that enables rapid detection of high-resolution three-dimensional (3D) light signals using only a single, low-format camera sensor area. SLIM pushes the boundaries of 3D optical microscopy, achieving over one thousand volumes per second across a large field of view of 550 μm in diameter and 300 μm in depth. Using SLIM, we demonstrated blood cell velocimetry across the embryonic zebrafish brain and in a free-moving tail exhibiting high-frequency swinging motion. The millisecond temporal resolution also enables accurate voltage imaging of neural membrane potentials in the leech ganglion. These results collectively establish SLIM as a versatile and robust imaging tool for high-speed microscopy applications.

show abstract

Section: Discussionmentioning

confidence: 99%

Kilohertz volumetric imaging of in-vivo dynamics using squeezed light field microscopy

Wang,

Zhao,

Wagenaar

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The incorporating of deep neural networks enlarges this design space of microscopy by introducing prior knowledge of high-resolution data 20,21 . Previous view-channel-depth light-field microscopy (VCD-LFM) is capable of directly reconstructing high-resolution 3D volume from 2D light-field (LF) raw data by splitting various views from LF and incorporating successive extracted features in network into multiple channels to yield 3D image stacks, successfully pushing the spatial resolution of LFM to diffraction limit and showing outstanding performance in imaging cellular structures [22][23][24] . However, learning the mapping function to reconstruct from 2D under-sampling LF to 3D super-resolution (SR) volumes is challenging for usual one-stage model, since the degradation model of light field imaging is an extremely intricate process coupling multiple resolution degradation and noise and compressing the space bandwidth product by ~500 times, resulting in the limited performance of reaching sub-diffraction-limited resolution with high fidelity.…”

Section: Introductionmentioning

confidence: 99%

“…We previously report view-channel-depth light-field microscopy (VCD-LFM), in which a VCD network model is trained to learn the nonlinear relationships between the 3D confocal ground truths and their 2D light-field projections and afterwards, can directly reconstruct high-resolution 3D volume from a single 2D light-field image through using the well-trained channels in the VCD model to transform the implicit features of the light-field views into depth information of a 3D stack. With deep-learning model combining the high-resolution advances of scanning microscopy into high-speed imaging of LFM, VCD-LFM holds the promise for high-speed and high-resolution 4D imaging of live samples [26][27][28] . However, current end-to-end supervised networks encounter constraints in term of both enhanced capabilities and generalization ability.…”

Section: Introductionmentioning

confidence: 99%

Sustained 3D isotropic imaging of subcellular dynamics using adaptive VCD light-field microscopy 2.0

Zhu

Sun

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

Long-term and high-speed 3D imaging of live cells beyond diffraction limit remains a big challenge for current super-resolution microscopy implementations, owing to their severe phototoxicity and limited volumetric imaging rate. While emerging light-field microscopy (LFM) has mitigated this issue through rapid and mild 3D imaging of dynamic biological processes with single 2D snapshots, it suffers from a suboptimal spatial resolution close to diffraction limit, which greatly compromises its applications for live-cell imaging. Here, we propose a super-resolution light-field reconstruction strategy based on an optics-aware view-channel-depth (VCD 2.0) deep-learning strategy. Through a stepwise inversion of the physical degradation process during light-filed imaging, VCD 2.0 strongly pushes the light-field microscopy beyond diffraction limit, and achieves instant 3D reconstructions from raw 2D light-field snapshots with an ~14-fold resolution improvement. After being combined with a simple-and-robust light-field add-on design, VCD 2.0 further enables long-term 3D super-resolution imaging (over 30000 volumes) of various intracellular dynamics (FOV of ~220*220*10 micro m3) even on an ordinary 2D fluorescence microscope. Using this novel and readily-accessible approach, we successfully capture the fast morphology changes of diverse organelles, such as the extension and constriction in lysosome, at a near isotropic resolution of ~120 nm, and a high volume rate up to 50 Hz.

show abstract

Multifocal optical projection microscopy enables label-free 3D measurement of cardiomyocyte cluster contractility

Belay,

Figueiras,

Hyttinen

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Human induced pluripotent stem cell (hiPSC)-derived cardiomyocyte (CM) models have become an attractive tool for in vitro cardiac disease modeling and drug studies. These models are moving towards more complex three-dimensional microphysiological organ-on-chip systems. Label-free imaging-based techniques capable of quantifying contractility in 3D are needed, as traditional two-dimensional methods are ill-suited for 3D applications. Here, we developed multifocal (MF) optical projection microscopy (OPM) by integrating an electrically tunable lens to our in-house built optical projection tomography setup for extended depth of field brightfield imaging in CM clusters. We quantified cluster biomechanics by implementing our previously developed optical flow-based CM video analysis for MF-OPM. To demonstrate, we acquired and analyzed multiangle and multifocal projection videos of beating hiPSC-CM clusters in 3D hydrogel. We further quantified cluster contractility response to temperature and adrenaline and observed changes to beating rate and relaxation. Challenges emerge from light penetration and overlaying textures in larger clusters. However, our findings indicate that MF-OPM is suitable for contractility studies of 3D clusters. Thus, for the first time, MF-OPM is used in CM studies and hiPSC-CM 3D cluster contraction is quantified in multiple orientations and imaging planes.

show abstract

A practical guide to deep-learning light-field microscopy for 3D imaging of biological dynamics

Cited by 5 publications

References 8 publications

Kilohertz volumetric imaging of in-vivo dynamics using squeezed light field microscopy

Kilohertz volumetric imaging of in-vivo dynamics using squeezed light field microscopy

Sustained 3D isotropic imaging of subcellular dynamics using adaptive VCD light-field microscopy 2.0

Multifocal optical projection microscopy enables label-free 3D measurement of cardiomyocyte cluster contractility

Contact Info

Product

Resources

About