System-level analysis and design for RGB-NIR CMOS camera

Abstract

This paper presents system-level analysis of a sensor capable of simultaneously acquiring both standard absorption based RGB color channels (400-700nm, ~75nm FWHM), as well as an additional NIR channel (central wavelength: ~808 nm, FWHM: ~30nm collimated light). Parallel acquisition of RGB and NIR info on the same CMOS image sensor is enabled by monolithic pixel-level integration of both a NIR pass thin film filter and NIR blocking filters for the RGB channels. This overcomes the need for a standard camera-level NIR blocking filter to remove the NIR leakage present in standard RGB absorption filters from ~700-1000nm. Such a camera-level NIR blocking filter would inhibit the acquisition of the NIR channel on the same sensor. Thin film filters do not operate in isolation. Rather, their performance is influenced by the system context in which they operate. The spectral distribution of light arriving at the photo diode is shaped a.o. by the illumination spectral profile, optical component transmission characteristics and sensor quantum efficiency. For example, knowledge of a low quantum efficiency (QE) of the CMOS image sensor above 800nm may reduce the filter's blocking requirements and simplify the filter structure. Similarly, knowledge of the incoming light angularity as set by the objective lens' F/# and exit pupil location may be taken into account during the thin film's optimization. This paper demonstrates how knowledge of the application context can facilitate filter design and relax design trade-offs and presents experimental results.

Decoupling channel count from field of view and spatial resolution in single-sensor imaging systems for fluorescence image-guided surgery

“…Consequently, these single-sensor devices typically provide three channels in the visible and one channel in the near-infrared to maximize the field of view and spatial resolution while providing the bare minimum channel count. 17 – 27 …”

“… 28 with the four-channel imaging systems that have long been favored by researchers and manufacturers alike. 17 – 27 We do so by fixing the largest scale available to the camera (i.e., fixing the field of view) and evaluating performance at the smallest scale available to the camera (i.e., evaluating performance at the spatial resolution). A discussion of the two imaging systems is provided in Sec.…”

“…Consequently, these single-sensor devices typically provide three channels in the visible and one channel in the near-infrared to maximize the field of view and spatial resolution while providing the bare minimum channel count. [17][18][19][20][21][22][23][24][25][26][27] What emerges within this design space is a complex relationship between the parameters that would optimize an image-guided surgery and the parameters that are practical in an imaging system. Maximum benefit to both the surgeon and patient may not be achieved via the default implementations of division-of-time, division-of-optical-path, and division-of-focal-plane instruments but may instead necessitate careful modifications to these architectures.…”

Demosaicing of RGBW Color Filter Array Based on Rank Minimization with Colorization Constraint

“…Recently, the demand for high-sensitivity color images has increased for various products such as autonomous driving cars and surveillance cameras.Multispectral filter arrays have been recently proposed to capture extra information other than the three primary color channels to overcome the physical limitations of the RGB CFA. For example, near-infrared pixels have been incorporated into the RGB CFA, which results in the RGB-NIR CFA, to increase the contrast between objects in the scene and the optical depth [ 18 , 19 , 20 , 21 , 22 , 23 , 24 ]. However, normally, the color images reconstructed from RGB-NIR CFAs have lower SNR values than those reconstructed from RGB CFAs, as the correlation of the brightness values of the near-infrared pixels and those of the RGB pixels are not so strong.…”

Fabrication of a CMOS-based Imaging Chip with Monolithically Integrated RGB and NIR Filters

“…This paper describes a combination of both functionalities onto a single image sensor, for the first time combining narrowband pixelated interference filters with traditional RGB polymer based colour filters. This is obtained by extending a classical RGB imager with a NIR sensitive pixel, enabling parallel acquisition of four channels [2]. The cost of the system is kept low by using standard, off-the-shelve CMOS sensors, standard CMOS technology for the integration of the filter stacks and the shield, and finishing with standard polymeric RGB colour filter processing.…”