Untrained networks for compressive lensless photography

Abstract

Compressive lensless imagers enable novel applications in an extremely compact device, requiring only a phase or amplitude mask placed close to the sensor. They have been demonstrated for 2D and 3D microscopy, single-shot video, and single-shot hyperspectral imaging; in each case, a compressive-sensing-based inverse problem is solved in order to recover a 3D data-cube from a 2D measurement. Typically, this is accomplished using convex optimization and hand-picked priors. Alternatively, deep learning-based reconstruction methods offer the promise of better priors, but require many thousands of ground truth training pairs, which can be difficult or impossible to acquire. In this work, we propose an unsupervised approach based on untrained networks for compressive image recovery. Our approach does not require any labeled training data, but instead uses the measurement itself to update the network weights. We demonstrate our untrained approach on lensless compressive 2D imaging, single-shot high-speed video recovery using the camera’s rolling shutter, and single-shot hyperspectral imaging. We provide simulation and experimental verification, showing that our method results in improved image quality over existing methods.

Learning-Based Image Transport Through Disordered Optical Fibers With Transverse Anderson Localization

“…Limited by the unique imaging objects and environments, it is quite challenging to access the distal end of the imaging unit and acquire labeled training data. To resolve these issues, unsupervised or semi-supervised learning approaches might be able to provide a new avenue for future systems in that they do not need strictly labeled data [127,[145][146][147]. This would release the demanding requirements on the amount of necessary training data and time as well as reduce the heavy burden on system calibrations.…”

“…This would release the demanding requirements on the amount of necessary training data and time as well as reduce the heavy burden on system calibrations. To implement unsupervised or semisupervised learning, integrating the physics modeling with the DCNN architecture is shown to be a wise choice [127,146]. Besides, recently fast-growing generative adversarial networks would be able to provide another potential solution to fiber imaging systems as well [148].…”

Untrained Neural Network Priors for Inverse Imaging Problems: A Survey

“…A complete task-specific physical model (representing dualwavelength in-line holography) was incorporated in the UNNP framework to help the network effectively eliminate the effect of amplified noise. Monakhova et al investigated both under-parameterized and over-parameterized UNNPs for compressed sensing based lensless 2D imaging [44]. They showed that over-parameterized UNNPs provide superior performance compared to under-parameterized networks.…”

Coded Illumination for 3D Lensless Imaging

“…Popular methods include adding sparse and low-rank priors [15]- [19] or natural image prior [20]- [22]. Recently, a number of methods have been proposed that use deep networks to reconstruct or post-process the images from lensless measurements [23]- [26]. Some of these methods provide exceptional improvement over traditional optimization-based methods.…”