Digital pixel CMOS focal plane array with on-chip multiply accumulate units for low-latency image processing

Abstract

A digital pixel CMOS focal plane array has been developed to enable low latency implementations of image processing systems such as centroid trackers, Shack-Hartman wavefront sensors, and Fitts correlation trackers through the use of in-pixel digital signal processing (DSP) and generic parallel pipelined multiply accumulate (MAC) units. Light intensity digitization occurs at the pixel level, enabling in-pixel DSP and noiseless data transfer from the pixel array to the peripheral processing units. The pipelined processing of row and column image data prior to off chip readout reduces the required output bandwidth of the image sensor, thus reducing the latency of computations necessary to implement various image processing systems. Data volume reductions of over 80% lead to sub 10µs latency for completing various tracking and sensor algorithms. This paper details the architecture of the pixel-processing imager (PPI) and presents some initial results from a prototype device fabricated in a standard 65nm CMOS process hybridized to a commercial off-the-shelf short-wave infrared (SWIR) detector array.

Smart pixel imaging with computational-imaging arrays

“…21 SWIR-PPI is hybridized to a COTS InGaAs detector array via indium bump bonding. Each pixel in the DROIC includes a sigma-delta ADC capable of recording up to 28 bits of inpixel digital output.…”

Optical design and characterization of an advanced computational imaging system

“…[12][13][14][15][16][17] We have previously shown the utility of MIT LL focal plane arrays for high-speed transient imaging of hyper-temporal targets by leveraging inpixel flutter shutter capabilities combined with on-chip processing. [18][19][20] Our PETEI testbed combines dynamic optical encoding at the pupil plane with novel focal plane array on-chip and per-pixel flutter shutter to realize image plane coding with no mechanical moving parts. PETEI is used to realize applications such as passive 3D ranging, pixel superresolution, and extended depth of focus with the addition of a compressive sensing framework.…”