PhaseCam3D — Learning Phase Masks for Passive Single View Depth Estimation

Wu, Yi‐Cheng; Boominathan, Vivek; Chen, Huaijin; Sankaranarayanan, Aswin C.; Veeraraghavan, Ashok

doi:10.1109/iccphot.2019.8747330

Cited by 106 publications

(100 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Deep learning has proven to be adept at analyzing microscopic data [ 21 – 29 ], especially for single-molecule localization, handling dense fields of emitters over small axial ranges (<1.5 μ m) [ 19 , 30 – 37 ] or sparse emitters spread over larger ranges [ 38 ]. Moreover, an emerging application is to jointly design the optical system alongside the data processing algorithm, enabling end-to-end optimization of both components [ 36 , 39 – 46 ]. Here we present DeepSTORM3D, consisting of two fundamental contributions to high-density 3D localization microscopy over large axial ranges.…”

Section: Introductionmentioning

confidence: 99%

DeepSTORM3D: dense 3D localization microscopy and PSF design by deep learning

et al. 2020

View full text Add to dashboard Cite

Localization microscopy is an imaging technique in which the positions of individual point emitters (e.g. fluorescent molecules) are precisely determined from their images. This is a key ingredient in single/multiple-particle-tracking and super-resolution microscopy. Localization in three-dimensions (3D) can be performed by modifying the image that a point-source creates on the camera, namely, the point-spread function (PSF). The PSF is engineered to vary distinctively with emitter depth, using additional optical elements. However, localizing multiple adjacent emitters in 3D poses a significant algorithmic challenge, due to the lateral overlap of their PSFs. Here, we train a neural network to localize multiple emitters with densely overlapping PSFs over a large axial range. Furthermore, we then use the network to design the optimal PSF for the multi-emitter case. We demonstrate our approach experimentally with super-resolution reconstructions of mitochondria and volumetric imaging of fluorescently labeled telomeres in cells.

show abstract

Section: Introductionmentioning

confidence: 99%

DeepSTORM3D: dense 3D localization microscopy and PSF design by deep learning

et al. 2020

View full text Add to dashboard Cite

show abstract

“…End-to-end Optics Design Joint optimization of optics and reconstruction has demonstrated superior performance over traditional heuristic approaches in color image restoration [4], microscopy [25,32,49,59], monocular depth imaging [5,17,21,65], super-resolution and extended depth of field [60], and time-of-flight imaging [38,61].…”

Section: Related Workmentioning

confidence: 99%

Learning Rank-1 Diffractive Optics for Single-Shot High Dynamic Range Imaging

Sun

Tseng

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

High-dynamic range (HDR) imaging is an essential imaging modality for a wide range of applications in uncontrolled environments, including autonomous driving, robotics, and mobile phone cameras. However, existing HDR techniques in commodity devices struggle with dynamic scenes due to multi-shot acquisition and postprocessing time, e.g. mobile phone burst photography, making such approaches unsuitable for real-time applications. In this work, we propose a method for snapshot HDR imaging by learning an optical HDR encoding in a single image which maps saturated highlights into neighboring unsaturated areas using a diffractive optical element (DOE). We propose a novel rank-1 parameterization of the DOE which drastically reduces the optical search space while allowing us to efficiently encode high-frequency detail. We propose a reconstruction network tailored to this rank-1 parametrization for the recovery of clipped information from the encoded measurements. The proposed end-to-end framework is validated through simulation and real-world experiments and improves the PSNR by more than 7 dB over state-ofthe-art end-to-end designs.

show abstract

“…We chose to create this PSF using a phase mask ( Fig. 1b), because they are capable of producing a wide variety of PSFs [33][34][35][36][37][38] and allow higher light throughput compared to amplitude masks (which block some of the incoming light) 21 . The second major innovation in FlatScope 2.0 is a compact light delivery and filtering system that enables epifluorescence imaging.…”

Section: Introductionmentioning

confidence: 99%

In vivofluorescence imaging with a flat, lensless microscope

Adams

Boominathan

Gao

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Fluorescence imaging over large areas of the brain in freely behaving animals would allow researchers to better understand the relationship between brain activity and behavior; however, traditional microscopes capable of high spatial resolution and large fields of view (FOVs) require large and heavy lenses that restrict animal movement. While lensless imaging has the potential to achieve both high spatial resolution and large FOV with a thin lightweight device, lensless imaging has yet to be achieved in vivo due to two principal challenges: (a) biological tissue typically has lower contrast than resolution targets, and (b) illumination and filtering must be integrated into this non-traditional device architecture. Here, we show that in vivo fluorescence imaging is possible with a thin lensless microscope by optimizing the phase mask and computational reconstruction algorithms, and integrating fiber optic illumination and thin-film color filters. The result is a flat, lensless imager that achieves better than 10 μm spatial resolution and a FOV that is 30× larger than other cellular resolution miniature microscopes.

show abstract

PhaseCam3D — Learning Phase Masks for Passive Single View Depth Estimation

Cited by 106 publications

References 38 publications

DeepSTORM3D: dense 3D localization microscopy and PSF design by deep learning

DeepSTORM3D: dense 3D localization microscopy and PSF design by deep learning

Learning Rank-1 Diffractive Optics for Single-Shot High Dynamic Range Imaging

In vivofluorescence imaging with a flat, lensless microscope

Contact Info

Product

Resources

About