Abstract

We describe an advanced computational imaging system with an optical architecture that enables simultaneous and dynamic pupil-plane and image-plane coding accommodating several task-specific applications. We assess the optical requirement trades associated with custom and commercial-off-the-shelf (COTS) optics and converge on the development of two low-cost and robust COTS testbeds. The first is a coded-aperture programmable pixel imager employing a digital micromirror device (DMD) for image plane per-pixel oversampling and spatial super-resolution experiments. The second is a simultaneous pupil-encoded and time-encoded imager employing a DMD for pupil apodization or a deformable mirror for wavefront coding experiments. These two testbeds are built to leverage two MIT Lincoln Laboratory focal plane arrays-an orthogonal transfer CCD with non-uniform pixel sampling and on-chip dithering and a digital readout integrated circuit (DROIC) with advanced on-chip per-pixel processing capabilities. This paper discusses the derivation of optical component requirements, optical design metrics, and performance analyses for the two testbeds built.

Smart pixel imaging with computational-imaging arrays (SPICA) transfers image plane coding typically realized in the optical architecture to the digital domain of the focal plane array, thereby minimizing signal-to-noise losses associated with static filters or apertures and inherent diffraction concerns. MIT Lincoln Laboratory has been developing digitalpixel focal plane array (DFPA) devices for many years. In this work, we leverage legacy designs modified with new features to realize a computational imaging array (CIA) with advanced pixel-processing capabilities. We briefly review the use of DFPAs for on-chip background removal and image plane filtering. We focus on two digital readout integrated circuits (DROICS) as CIAs for two-dimensional (2D) transient target tracking and three-dimensional (3D) transient target estimation using per-pixel coded-apertures or flutter shutters. This paper describes two DROICs -a SWIR pixelprocessing imager (SWIR-PPI) and a Visible CIA (VISCIA). SWIR-PPI is a DROIC with a 1 kHz global frame rate with a maximum per-pixel shuttering rate of 100 MHz, such that each pixel can be modulated by a time-varying, pseudorandom, and duo-binary signal (+1,-1,0). Combining per-pixel time-domain coding and processing enables 3D (x,y,t) target estimation with limited loss of spatial resolution. We evaluate structured and pseudo-random encoding strategies and employ linear inversion and non-linear inversion using total-variation minimization to estimate a 3D data cube from a single 2D temporally-encoded measurement. The VISCIA DROIC, while low-resolution, has a 6 kHz global frame rate and simultaneously encodes eight periodic or aperiodic transient target signatures at a maximum rate of 50 MHz using eight 8-bit counters. By transferring pixel-based image plane coding to the DROIC and utilizing sophisticated processing, our CIAs enable on-chip temporal super-resolution.

Abstract. This paper presents a novel super-resolution framework by exploring the properties of non-conventional pixel layouts and shapes. We show that recording multiple images, transformed in the octic group, with a sensor of asymmetric sub-pixel layout increases the spatial sampling compared to a conventional sensor with a rectilinear grid of pixels and hence increases the image resolution. We further prove a theoretical bound for achieving well-posed super-resolution with a designated magnification factor w.r.t. the number and distribution of sub-pixels. We also propose strategies for selecting good sub-pixel layouts and effective super-resolution algorithms for our setup. The experimental results validate the proposed theory and solution, which have the potential to guide the future CCD layout design with super-resolution functionality.

No abstract

A digital pixel CMOS focal plane array has been developed to enable low latency implementations of image processing systems such as centroid trackers, Shack-Hartman wavefront sensors, and Fitts correlation trackers through the use of in-pixel digital signal processing (DSP) and generic parallel pipelined multiply accumulate (MAC) units. Light intensity digitization occurs at the pixel level, enabling in-pixel DSP and noiseless data transfer from the pixel array to the peripheral processing units. The pipelined processing of row and column image data prior to off chip readout reduces the required output bandwidth of the image sensor, thus reducing the latency of computations necessary to implement various image processing systems. Data volume reductions of over 80% lead to sub 10µs latency for completing various tracking and sensor algorithms. This paper details the architecture of the pixel-processing imager (PPI) and presents some initial results from a prototype device fabricated in a standard 65nm CMOS process hybridized to a commercial off-the-shelf short-wave infrared (SWIR) detector array.

No abstract

No abstract