Smart pixel imaging with computational-imaging arrays

Abstract

Smart pixel imaging with computational-imaging arrays (SPICA) transfers image plane coding typically realized in the optical architecture to the digital domain of the focal plane array, thereby minimizing signal-to-noise losses associated with static filters or apertures and inherent diffraction concerns. MIT Lincoln Laboratory has been developing digitalpixel focal plane array (DFPA) devices for many years. In this work, we leverage legacy designs modified with new features to realize a computational imaging array (CIA) with advanced pixel-processing capabilities. We briefly review the use of DFPAs for on-chip background removal and image plane filtering. We focus on two digital readout integrated circuits (DROICS) as CIAs for two-dimensional (2D) transient target tracking and three-dimensional (3D) transient target estimation using per-pixel coded-apertures or flutter shutters. This paper describes two DROICs -a SWIR pixelprocessing imager (SWIR-PPI) and a Visible CIA (VISCIA). SWIR-PPI is a DROIC with a 1 kHz global frame rate with a maximum per-pixel shuttering rate of 100 MHz, such that each pixel can be modulated by a time-varying, pseudorandom, and duo-binary signal (+1,-1,0). Combining per-pixel time-domain coding and processing enables 3D (x,y,t) target estimation with limited loss of spatial resolution. We evaluate structured and pseudo-random encoding strategies and employ linear inversion and non-linear inversion using total-variation minimization to estimate a 3D data cube from a single 2D temporally-encoded measurement. The VISCIA DROIC, while low-resolution, has a 6 kHz global frame rate and simultaneously encodes eight periodic or aperiodic transient target signatures at a maximum rate of 50 MHz using eight 8-bit counters. By transferring pixel-based image plane coding to the DROIC and utilizing sophisticated processing, our CIAs enable on-chip temporal super-resolution.

A review of snapshot multidimensional optical imaging: Measuring photon tags in parallel

“…The second strategy is based on passive imaging and therefore does not need a specialized light source. Within this category, representative techniques are parallel streak imaging using a tilted lenslet array [200], temporal pixel multiplexing [201], compressed ultrafast photography [27], coded aperture compressive temporal imaging [202], programmable pixel compressive imaging [13], and smart pixel imaging with computational-imaging arrays [203, 204]. …”

“…By contrast, smart pixel imaging (SPI) with computational-imaging arrays [203, 204] transfers this encoding process to the digital domain by using a digital-pixel focal plane array, thereby minimizing the signal-to-noise loss caused by physical encoding elements, such as the DMD in CUP and P2C2, and the absorption mask in CACTI. In SPI, each detector pixel can be modulated by a time-varying, pseudo-random, and dual-binary signal (−1,1 or 1,0) at a rate up to 100 MHz.…”

“…However, in SPI the time-varying spatial encoding C ( t ) is introduced in the digital domain, rather than in the real image domain as in P2C2. Fernandez-Cull et al demonstrated that by employing algorithms such as TwIST, the event datacube I ( x,y,t ) can be reasonably estimated [203]. …”

Optical design and characterization of an advanced computational imaging system

“…[12][13][14][15][16][17] We have previously shown the utility of MIT LL focal plane arrays for high-speed transient imaging of hyper-temporal targets by leveraging inpixel flutter shutter capabilities combined with on-chip processing. [18][19][20] Our PETEI testbed combines dynamic optical encoding at the pupil plane with novel focal plane array on-chip and per-pixel flutter shutter to realize image plane coding with no mechanical moving parts. PETEI is used to realize applications such as passive 3D ranging, pixel superresolution, and extended depth of focus with the addition of a compressive sensing framework.…”

“…[23][24] The PETEI ACIS testbed draws upon previous work for on-chip coded-aperture temporal imaging utilizing a digital readout integrated circuit (DROIC) with a per-pixel shutter for temporal super-resolution. 19 A major advantage associated with dynamic digital in-pixel shuttering includes flux preservation when a time-varying binary (1,-1) aperture is utilized versus typically used static apertures placed along the optical axis. Finally, DROIC pixel processing imagers (PPIs) re-purposed as CIAs provide inpixel analog-to-digital conversion and per-pixel processing allowing for novel spatial and temporal filtering and dynamic range enhancements as compared to conventional 2D focal plane arrays.…”