FLIP-Q: A QCIF Resolution Focal-Plane Array for Low-Power Image Processing

Abstract

This paper reports a 176×144-pixel smart image sensor designed and fabricated in a 0.35µm CMOS-OPTO process. The chip implements a massively parallel focal-plane processing array which can output different simplified representations of the scene at very low power. The array is composed of pixel-level processing elements which carry out analog image processing concurrently with photosensing. These processing elements can be grouped into fully-programmable rectangular-shape areas by loading the appropiate interconnection patterns into the registers at the edge of the array. The targeted processing can be thus performed block-wise. Readout is done pixel-by-pixel in a random access fashion. On-chip 8b ADC is provided. The image processing primitives implemented by the chip, experimentally tested and fully functional, are scale space and Gaussian pyramid generation, fully-programmable multiresolution scene representation -including foveation -and block-wise energy-based scene representation. The power consumption associated to the capture, processing and A/D conversion of an image flow at 30fps, with full-frame processing but reduced frame size output, ranges from 2.7mW to 5.6mW, depending on the operation to be performed.

Smart imaging for power-efficient extraction of Viola-Jones local descriptors

“…This voltage is repeatedly copied into V Sij by enabling the analog buffer through the control signal CP EN and then squared at V SQij in order to respectively support the computation of the integral and square integral images. We re-use the squarer reported in 19 because of its simplicity and successful experimental verification. The sum of pixels and squared pixels is carried out through charge redistribution enabled by the switches controlled by EN Si,i+1 , EN Sj,j+1 , EN SQi,i+1 and EN SQj,j+1 .…”

Form factor improvement of smart-pixels for vision sensors through 3-D vertically-integrated technologies

“…Fig.4 shows the pixel schematics and illustrative processing for a smart-CIS which employs mixed-signal MFPS to realize programmable spatial-temporal filtering [9]. It employs 22nJ/ cycle to complete binomial filtering operations at nsec rate which makes it very well suited for the front-end of portable vision-enabled wireless sensors.…”

“…The target of the processing illustrated in the figure is segmenting the zones of the image with the largest changes of intensity, that is, the relative values among the blocks of the scene representation are the key point here. It is done by the chip in [9] by in-pixel energy computation to guide subsequent foveation.…”

“…Circuit embedding can actually be made at different levels, namely: per-pixel [5]- [9], per-column [10], and per-chip [11]. The intensity of embedding at each level depend on the targeted applications.…”

Focal plane generation of multi-resolution and multi-scale image representation for low-power vision applications

“…Bearing in mind this type of applications, we have designed a prototype vision chip called FLIP-Q, reported recently. 9 This chip also follows the guidelines above sketched but only a reduced subset of focal-plane processing primitives is implemented. These primitives deliver user-defined simplifications of the scene at ultra low energy cost.…”