Video-rate hyperspectral camera based on a CMOS-compatible random array of Fabry–Pérot filters

Abstract

Hyperspectral (HS) imaging provides rich spatial and spectral information and extends image inspection beyond human perception. Existing approaches, however, suffer from several drawbacks such as low sensitivity, resolution and/or frame rate, which confines HS cameras to scientific laboratories. Here we develop a video-rate HS camera capable of collecting spectral information on real-world scenes with sensitivities and spatial resolutions comparable with those of a typical RGB camera. Our camera uses compressive sensing, whereby spatial–spectral encoding is achieved with an array of 64 complementary metal–oxide–semiconductor (CMOS)-compatible Fabry–Pérot filters placed onto a monochromatic image sensor. The array affords high optical transmission while minimizing the reconstruction error in subsequent iterative image reconstruction. The experimentally measured sensitivity of 45% for visible light, the spatial resolution of 3 px for 3 dB contrast, and the frame rate of 32.3 fps at VGA resolution meet the requirements for practical use. For further acceleration, we show that AI-based image reconstruction affords operation at 34.4 fps and full high-definition resolution. By enabling practical sensitivity, resolution and frame rate together with compact size and data compression, our HS camera holds great promise for the adoption of HS technology in real-world scenarios, including consumer applications such as smartphones and drones.

mHealth hyperspectral learning for instantaneous spatiospectral imaging of hemodynamics

“…Other snapshot imaging technologies employ dispersion patterns or coded apertures projecting irradiance mixed with spatial and spectral information to further enhance the light collection efficiency and readout rate ( 26 , 27 , 35–37 ). Subsequently, the modulated projection comprising spatial and spectral information is reconstructed into a hypercube by utilizing computational algorithms such as compressed (or compressive) sensing ( 32 , 38 , 39 ), Fourier transformation ( 40 , 41 ), or deep learning ( 19 , 20 , 42–48 ).…”

Metasurface-based computational imaging: a review

“…Among encoding schemes for snapshot hyperspectral imaging, random spectral encoding, as the simplest method, has showcased its potential to deliver high-quality and compact hyperspectral imaging 42 , 43 . Beyond its coding efficiency, its resilience to manufacturing and systematic errors makes it suitable for real-world applications.…”

“…Among encoding schemes for snapshot hyperspectral imaging, random spectral encoding, as the simplest method, has showcased its potential to deliver high-quality and compact hyperspectral imaging. 42,43 Beyond its coding efficiency, its resilience to manufacturing and systematic errors makes it suitable for real-world applications. Specifically, a proposal has been made for regular-shaped metasurface units to achieve random spectral encoding at the visible wavelength, 2 as shown in Fig.…”

Highly multicolored light-emitting arrays for compressive spectroscopy

“…Compared to reconstructive microscale spectral imaging approaches that rely on the modulation of passive elements such as a single tuned photodetector, modulation of the active illumination enables faster acquisition of spectral image stacks with larger image sizes due to its compatibility with standard microscopes, which avoids the need for time-intensive spatial scanning. Similarly, our approach does not rely on modification of or adaptation of the spectral element to the camera or image sensor (30)(31)(32). Because light emission can be generated across a broad spectral range, the potentially achievable spectral range is broader and does not rely on carefully engineered material syntheses or highly material-specific physics (9,33,34).…”