A snapshot multispectral imager with integrated tiled filters and optical duplication

Abstract

Although the potential of spectral imaging has been demonstrated in research environments, its adoption by industry has so far been limited due to the lack of high speed, low cost and compact spectral cameras. We have previously presented work to overcome this limitation by monolithically integrating optical interference filters on top of standard CMOS image sensors for high resolution pushbroom hyperspectral cameras. These cameras require a scanning of the scene and therefore introduce operator complexity due to the need for synchronization and alignment of the scanning to the camera. This typically leads to problems with motion blur, reduced SNR in high speed applications and detection latency and overall restricts the types of applications that can use this system. This paper introduces a novel snapshot multispectral imager concept based on optical filters monolithically integrated on top of a standard CMOS image sensor. By using monolithic integration for the dedicated, high quality spectral filters at its core, it enables the use of mass-produced fore-optics, reducing the total system cost. It overcomes the problems mentioned for scanning applications by snapshot acquisition, where an entire multispectral data cube is sensed at one discrete point in time. This is achieved by applying a novel, tiled filter layout and an optical sub-system which simultaneously duplicates the scene onto each filter tile. Through the use of monolithically integrated optical filters it retains the qualities of compactness, low cost and high acquisition speed, differentiating it from other snapshot spectral cameras based on heterogeneously integrated custom optics. Moreover, thanks to a simple cube assembly process, it enables real-time, low-latency operation. Our prototype camera can acquire multispectral image cubes of 256x256 pixels over 32 bands in the spectral range of 600-1000nm at a speed of about 30 cubes per second at daylight conditions up to 340 cubes per second at higher illumination levels as typically used in machine vision applications.

System-level analysis and design for RGB-NIR CMOS camera

“…Monolithic integration of Fabry-Pérot based interference filters has already been successfully demonstrated in [6], [7] which present the performance of very compact CMOS-based hyperspectral camera architectures developed at imec. Moreover, a multispectral CMOS image sensor based on monolithic pixel-level integration of interference filters was reported in [8] .…”

A tiny VIS-NIR snapshot multispectral camera

“…We have previously enabled fast, compact and cost-effective spectral cameras by monolithically integrating optical interference filters on top of standard CMOS image sensors for high resolution pushbroom hyperspectral cameras [19] and for non-scanning snapshot multispectral cameras with optical duplication [20] and per-pixel filter mosaics [21]. Through the monolithic integration of the filter on the sensor, the stray light in the system is heavily reduced 5.5kg and the sensitivity and the speed of the sensor are increased [22].…”

Energy balance in Spectral Filter Array camera design

“…For the analysis, we will consider the case of a camera based on N = [ 3,5,8,12] bands. We chose a three-band sensor to represent the typical color case, five bands are chosen for multispectral color sensors or colorimeters [44].…”

“…The filters considered in SFAs are most often a realization of a Fabry-Pérot interferometers today [5,7,37]. Future technical development may give rise to nano-tube or nano-hole SFA realizations [10].…”

“…Each spatial position of the array captures only one spectral channel similarly to the Bayer filters [4], and the filter transmission characteristics determine the sensor spectral properties. Several realizations of such setup have been shown recently in the literature [5][6][7]. An *Correspondence: pierre-jean.lapray@uha.fr Université de Bourgogne Franche Comté, LE2I, UMR CNRS 6306, 9 av.…”