Lensless Imaging with an End-to-End Deep Neural Network

Bae, Donggeon; Jung, Jaewoo; Baek, Nakkyu; Lee, Seung Ah

doi:10.1109/icce-asia49877.2020.9276865

Cited by 3 publications

(3 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some lensless cameras employ end-to-end deep neural network image reconstruction algorithms in the imaging process. While these systems do not undergo an explicit calibration phase, calibration is inherently integrated into the deep neural network's training regimen [27][28][29].…”

Section: Calibrationmentioning

confidence: 99%

Lensless Image Restoration Based on Multi-Stage Deep Neural Networks and Pix2pix Architecture

Liu,

Su,

Yao

et al. 2023

Photonics

View full text Add to dashboard Cite

Lensless imaging represents a significant advancement in imaging technology, offering unique benefits over traditional optical systems due to its compact form factor, ideal for applications within the Internet of Things (IoT) ecosystem. Despite its potential, the intensive computational requirements of current lensless imaging reconstruction algorithms pose a challenge, often exceeding the resource constraints typical of IoT devices. To meet this challenge, a novel approach is introduced, merging multi-level image restoration with the pix2pix generative adversarial network architecture within the lensless imaging sphere. Building on the foundation provided by U-Net, a Multi-level Attention-based Lensless Image Restoration Network (MARN) is introduced to further augment the generator’s capabilities. In this methodology, images reconstructed through Tikhonov regularization are perceived as degraded images, forming the foundation for further refinement via the Pix2pix network. This process is enhanced by incorporating an attention-focused mechanism in the encoder--decoder structure and by implementing stage-wise supervised training within the deep convolutional network, contributing markedly to the improvement of the final image quality. Through detailed comparative evaluations, the superiority of the introduced method is affirmed, outperforming existing techniques and underscoring its suitability for addressing the computational challenges in lensless imaging within IoT environments. This method can produce excellent lensless image reconstructions when sufficient computational resources are available, and it consistently delivers optimal results across varying computational resource constraints. This algorithm enhances the applicability of lensless imaging in applications such as the Internet of Things, providing higher-quality image acquisition and processing capabilities for these domains.

show abstract

Section: Calibrationmentioning

confidence: 99%

Lensless Image Restoration Based on Multi-Stage Deep Neural Networks and Pix2pix Architecture

Liu,

Su,

Yao

et al. 2023

Photonics

View full text Add to dashboard Cite

show abstract

“…Fortunately, lensless computational cameras have been developed as an alternative, which place thin masks other than lens in front of the sensors, thereby significantly decreasing the thickness and weight of camera. Considerable algorithms have been proposed to restore images from the captured coded measurements [1][2][3]. However, compared with images directly captured by lens-based cameras, the reconstructed images from lensless cameras inevitably have various degradations (e.g., blurring, lower resolution and low signal-to-noise ratio).…”

Section: Introductionmentioning

confidence: 99%

Text detection and recognition based on a lensless imaging system

zhang

Lin

et al. 2022

Appl. Opt.

View full text Add to dashboard Cite

Lensless cameras are characterized by several advantages (e.g., miniaturization, ease of manufacture, and low cost) as compared with conventional cameras. However, they have not been extensively employed due to their poor image clarity and low image resolution, especially for tasks that have high requirements on image quality and details such as text detection and text recognition. To address the problem, a framework of deep-learning-based pipeline structure was built to recognize text with three steps from raw data captured by employing lensless cameras. This pipeline structure consisted of the lensless imaging model U-Net, the text detection model connectionist text proposal network (CTPN), and the text recognition model convolutional recurrent neural network (CRNN). Compared with the method focusing only on image reconstruction, U-Net in the pipeline was able to supplement the imaging details by enhancing factors related to character categories in the reconstruction process, so the textual information can be more effectively detected and recognized by CTPN and CRNN with fewer artifacts and high-clarity reconstructed lensless images. By performing experiments on datasets of different complexities, the applicability to text detection and recognition on lensless cameras was verified. This study reasonably demonstrates text detection and recognition tasks in the lensless camera system, and develops a basic method for novel applications.

show abstract

“…ADMM [Antipa et al 2018] and FISTA [Beck and Teboulle 2009]). To address this, a growing number of works use data-driven Convolutional Neural Networks (CNNs) to improve the speed and quality of lensless image reconstructions [Bae et al 2020; Barbastathis et al 2019;Sinha et al 2017]. A typical CNN with a limited receptive field size fails to accurately model the light transport of the imaging system [Goodman 2005], leading to learned models which fail to reconstruct lensless images accurately and efficiently.…”

Section: Introductionmentioning

confidence: 99%

Unrolled Primal-Dual Networks for Lensless Cameras

Oliver¹,

Antipa²,

Bostan³

et al. 2022

Preprint

View full text Add to dashboard Cite

Fig. 1. Comparison of our unrolled primal-dual network with state of the art. Intensive post-processing of lensless images cannot correct the model error, over-smoothing images and removing important features, such as text. We propose to replace classical lensless reconstruction methods with our physicallyinformed unrolled primal-dual model, where the model includes a series of learned forward and adjoint models (pseudo point-spread functions and their inverse). As a result, our work can produce plausible images and recover additional features while reducing the need for deep post-processing networks such as U-Nets [Ronneberger et al. 2015] (Source image courtesy MIR Flickr [Huiskes and Lew 2008]).Conventional image reconstruction models for lensless cameras often assume that each measurement results from convolving a given scene with a single experimentally measured point-spread function. These image reconstruction models fall short in simulating lensless cameras truthfully as these models are not sophisticated enough to account for optical aberrations or scenes with depth variations. Our work shows that learning a supervised primal-dual reconstruction method results in image quality matching state of the art in the literature without demanding a large network capacity. This improvement stems from our primary finding that embedding learnable forward and adjoint models in a learned primal-dual optimization framework can even improve the quality of reconstructed images (+5dB PSNR

show abstract

Lensless Imaging with an End-to-End Deep Neural Network

Cited by 3 publications

References 5 publications

Lensless Image Restoration Based on Multi-Stage Deep Neural Networks and Pix2pix Architecture

Lensless Image Restoration Based on Multi-Stage Deep Neural Networks and Pix2pix Architecture

Text detection and recognition based on a lensless imaging system

Unrolled Primal-Dual Networks for Lensless Cameras

Contact Info

Product

Resources

About