Text detection and recognition based on a lensless imaging system

Self Cite

A lensless camera is an imaging system that uses a mask in place of a lens, making it thinner, lighter, and less expensive than a lensed camera. However, additional complex computation and time are required for image reconstruction. This work proposes a deep learning model named Raw3dNet that recognizes hand gestures directly on raw videos captured by a lensless camera without the need for image restoration. In addition to conserving computational resources, the reconstruction-free method provides privacy protection. Raw3dNet is a novel end-to-end deep neural network model for the recognition of hand gestures in lensless imaging systems. It is created specifically for raw video captured by a lensless camera and has the ability to properly extract and combine temporal and spatial features. The network is composed of two stages: 1. spatial feature extractor (SFE), which enhances the spatial features of each frame prior to temporal convolution; 2. 3D-ResNet, which implements spatial and temporal convolution of video streams. The proposed model achieves 98.59% accuracy on the Cambridge Hand Gesture dataset in the lensless optical experiment, which is comparable to the lensed-camera result. Additionally, the feasibility of physical object recognition is assessed. Further, we show that the recognition can be achieved with respectable accuracy using only a tiny portion of the original raw data, indicating the potential for reducing data traffic in cloud computing scenarios.

Section: Lensless Imagingmentioning

confidence: 99%

Hand gestures recognition in videos taken with a lensless camera

zhang¹,

Wu²,

Lin³

et al. 2022

Self Cite

Autofocusing of Fresnel zone aperture lensless imaging for QR code recognition

Liu¹,

Wu²,

Cao³

2023

Fresnel zone aperture (FZA) lensless imaging encodes the incident light into a hologram-like pattern, so that the scene image can be numerically focused at a long imaging range by the back propagation method. However, the target distance is uncertain. The inaccurate distance causes blurs and artifacts in the reconstructed images. This brings difficulties for the target recognition applications, such as quick response code scanning. We propose an autofocusing method for FZA lensless imaging. By incorporating the image sharpness metrics into the back propagation reconstruction process, the method can acquire the desired focusing distance and reconstruct noise-free high-contrast images. By combining the Tamura of the gradient metrics and nuclear norm of gradient, the relative error of estimated object distance is only 0.95% in the experiment. The proposed reconstruction method significantly improves the mean recognition rate of QR code from 4.06% to 90.00%. It paves the way for designing intelligent integrated sensors.

Dual-branch fusion model for lensless imaging

zhang¹,

Wu²,

Xu³

et al. 2023

A lensless camera is an imaging system that replaces the lens with a mask to reduce thickness, weight, and cost compared to a lensed camera. The improvement of image reconstruction is an important topic in lensless imaging. Model-based approach and pure data-driven deep neural network (DNN) are regarded as two mainstream reconstruction schemes. In this paper, the advantages and disadvantages of these two methods are investigated to propose a parallel dual-branch fusion model. The model-based method and the data-driven method serve as two independent input branches, and the fusion model is used to extract features from the two branches and merge them for better reconstruction. Two types of fusion model named Merger-Fusion-Model and Separate-Fusion-Model are designed for different scenarios, where Separate-Fusion-Model is able to adaptively allocate the weights of the two branches by the attention module. Additionally, we introduce a novel network architecture named UNet-FC into the data-driven branch, which enhances reconstruction by making full use of the multiplexing property of lensless optics. The superiority of the dual-branch fusion model is verified by drawing comparison with other state-of-the-art methods on public dataset (+2.95dB peak signal-to-noise (PSNR), +0.036 structural similarity index (SSIM), −0.0172 Learned Perceptual Image Patch Similarity (LPIPS)). Finally, a lensless camera prototype is constructed to further validate the effectiveness of our method in a real lensless imaging system.