A novel compressive sensing approach to simultaneously acquire color and near-infrared images on a single sensor

Abstract

Sensors of most digital cameras are made of silicon that is inherently sensitive to both the visible and near-infrared parts of the electromagnetic spectrum. In this paper, we address the problem of color and NIR joint acquisition. We propose a framework for the joint acquisition that uses only a single silicon sensor and a slightly modified version of the Bayer color-filter array that is already mounted in most color cameras. Implementing such a design for an RGB and NIR joint acquisition system requires minor changes to the hardware of commercial color cameras.One of the important differences between this design and the conventional color camera is the post-processing applied to the captured values to reconstruct full resolution images. By using a CFA similar to Bayer, the sensor records a mixture of NIR and one color channel in each pixel. In this case, separating NIR and color channels in different pixels is equivalent to solving an under-determined system of linear equations. To solve this problem, we propose a novel algorithm that uses the tools developed in the field of compressive sensing. Our method results in high-quality RGB and NIR images (the average PSNR of more than 30 dB for the reconstructed images) and shows a promising path towards RGB and NIR cameras.

Proof-of-concept and evaluation of a dual function visible/NIR camera for iris authentication in smartphones

“…Like any other commercial camera, the spectral sensitivities of red, green, and blue pixels in this camera without the hotmirror will have a peak around 850nm [8]. The NIR pixels will also have a similar peak value around 850nm.…”

Demultiplexing Visible and Near-Infrared Information in Single-Sensor Multispectral Imaging

“…Sparse decomposition methods are used both in the source separation literature [2,18] and in color and multispectral imaging [3,5,16]. Golbabaee et al use sparse decomposition to reconstruct multispectral data from a few noisy measurements [5].…”

“…Golbabaee et al use sparse decomposition to reconstruct multispectral data from a few noisy measurements [5]. In [16], we employed a sparsity-based estimation technique to capture color and NIR images (four-channel images) using the Bayer color filter array.…”

Near-Infrared Fusion for Photorealistic Image Dehazing

“…In most digital cameras, a filter is placed in front of the sensor to block the NIR radiation and stop it from affecting the acquisition of the color image [1]. Using recent research [2,3,4,5], a camera can simultaneously capture both visible and NIR images after removing the NIR-blocking filter. Image dehazing [6] is one of many applications where the additional NIR image proves to be very useful.…”