Towards Photorealistic Reconstruction of Highly Multiplexed Lensless Images

Khan, Salman Siddique; Adarsh, V R; Boominathan, Vivek; Tan, Jasper; Veeraraghavan, Ashok; Mitra, Kaushik

doi:10.1109/iccv.2019.00795

Cited by 29 publications

(21 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Later, in [7], Johnson et al first combined the advantages of the feed-forward neural networks and perceptual loss for style transfer and super-resolution. Subsequently, perceptual loss has been applied to many image-formation applications, including [47,48], etc. However, to our knowledge, perceptual loss has not yet been successfully applied to phase retrieval, neither has it been applied to inverse problems under extremely low light condition in general, as this paper is concerned.…”

Section: Training For Inverse Problems With Vgg Based Perceptual Lossmentioning

confidence: 99%

Probing shallower: perceptual loss trained Phase Extraction Neural Network (PLT-PhENN) for artifact-free reconstruction at low photon budget

Deng

Goy

et al. 2020

Opt. Express

View full text Add to dashboard Cite

show abstract

Section: Training For Inverse Problems With Vgg Based Perceptual Lossmentioning

confidence: 99%

Probing shallower: perceptual loss trained Phase Extraction Neural Network (PLT-PhENN) for artifact-free reconstruction at low photon budget

Deng

Goy

et al. 2020

Opt. Express

View full text Add to dashboard Cite

show abstract

“…• We propose an efficient implementation for the learnable intermediate stage of non-separable or general lensless model. In [7], we had only shown this for the separable lensless model. Here we non-trivially extend it to the general lensless case.…”

Section: Introductionmentioning

confidence: 94%

“…Although these methods provide interpretability, the drawbacks they offer include increased computation and higher memory consumption due to large number of unrolled iterations. The proposed method and its preliminary version [7] fall under the category of physics inspired deep neural network as well. However, they don't involve any unrolling thereby avoiding large computational and memory cost.…”

Section: Image Reconstructionmentioning

confidence: 99%

“…lensless mask based lensless cameras have inferior characteristics as compared to their existing non-separable counterparts. In our previous work [7], we had demonstrated FlatNet's effectiveness for separable lensless model. But it cannot be trivially used for non-separable mask based lensless cameras.…”

Section: Introductionmentioning

confidence: 99%

“…• Similar to the display and direct captured measurements collected using the separable mask FlatCam and described in our previous work [7], we collect corresponding datasets for the non-separable mask PhlatCam [2].…”

mentioning

confidence: 99%

See 2 more Smart Citations

FlatNet: Towards Photorealistic Scene Reconstruction from Lensless Measurements

Khan

Sundar

Boominathan

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

Lensless imaging has emerged as a potential solution towards realizing ultra-miniature cameras by eschewing the bulky lens in a traditional camera. Without a focusing lens, the lensless cameras rely on computational algorithms to recover the scenes from multiplexed measurements. However, the current iterative-optimization-based reconstruction algorithms produce noisier and perceptually poorer images. In this work, we propose a non-iterative deep learning-based reconstruction approach that results in orders of magnitude improvement in image quality for lensless reconstructions. Our approach, called FlatNet, lays down a framework for reconstructing high-quality photorealistic images from mask-based lensless cameras, where the camera's forward model formulation is known. FlatNet consists of two stages: (1) an inversion stage that maps the measurement into a space of intermediate reconstruction by learning parameters within the forward model formulation, and (2) a perceptual enhancement stage that improves the perceptual quality of this intermediate reconstruction. These stages are trained together in an end-to-end manner. We show high-quality reconstructions by performing extensive experiments on real and challenging scenes using two different types of lensless prototypes: one which uses a separable forward model and another, which uses a more general non-separable cropped-convolution model. Our end-to-end approach is fast, produces photorealistic reconstructions, and is easy to adopt for other mask-based lensless cameras.

show abstract

GEOMScope: Large Field‐of‐View 3D Lensless Microscopy with Low Computational Complexity

Tian

Yang

2021

Laser & Photonics Reviews

View full text Add to dashboard Cite

Imaging systems with miniaturized device footprint, real‐time processing speed, and high‐resolution 3D visualization are critical to broad biomedical applications such as endoscopy. Most of existing imaging systems rely on bulky lenses and mechanically refocusing to perform 3D imaging. Here, GEOMScope, a lensless single‐shot 3D microscope that forms image through a single layer of thin microlens array and reconstructs objects through an innovative algorithm combining geometrical‐optics‐based pixel back projection and background suppressions, is demonstrated. The effectiveness of GEOMScope is verified on resolution target, fluorescent particles, and volumetric objects. Comparing to other widefield lensless imaging devices, the required computational resource is significantly reduced, and the reconstruction speed is increased by orders of magnitude. This enables to image and recover large volume 3D object in high resolution with near real‐time processing speed. Such a low computational complexity is attributed to the joint design of imaging optics and reconstruction algorithms, and a joint application of geometrical optics and machine learning in the 3D reconstruction. More broadly, the excellent performance of GEOMScope in imaging resolution, volume, and reconstruction speed implicates that geometrical optics can greatly benefit and play an important role in computational imaging.

show abstract

Towards Photorealistic Reconstruction of Highly Multiplexed Lensless Images

Cited by 29 publications

References 27 publications

Probing shallower: perceptual loss trained Phase Extraction Neural Network (PLT-PhENN) for artifact-free reconstruction at low photon budget

Probing shallower: perceptual loss trained Phase Extraction Neural Network (PLT-PhENN) for artifact-free reconstruction at low photon budget

FlatNet: Towards Photorealistic Scene Reconstruction from Lensless Measurements

GEOMScope: Large Field‐of‐View 3D Lensless Microscopy with Low Computational Complexity

Contact Info

Product

Resources

About