Focal-Plane Sensing-Processing: A Power-Efficient Approach for the Implementation of Privacy-Aware Networked Visual Sensors

Abstract

The capture, processing and distribution of visual information is one of the major challenges for the paradigm of the Internet of Things. Privacy emerges as a fundamental barrier to overcome. The idea of networked image sensors pervasively collecting data generates social rejection in the face of sensitive information being tampered by hackers or misused by legitimate users. Power consumption also constitutes a crucial aspect. Images contain a massive amount of data to be processed under strict timing requirements, demanding high-performance vision systems. In this paper, we describe a hardware-based strategy to concurrently address these two key issues. By conveying processing capabilities to the focal plane in addition to sensing, we can implement privacy protection measures just at the point where sensitive data are generated. Furthermore, such measures can be tailored for efficiently reducing the computational load of subsequent processing stages. As a proof of concept, a full-custom QVGA vision sensor chip is presented. It incorporates a mixed-signal focal-plane sensing-processing array providing programmable pixelation of multiple image regions in parallel. In addition to this functionality, the sensor exploits reconfigurability to implement other processing primitives, namely block-wise dynamic range adaptation, integral image computation and multi-resolution filtering. The proposed circuitry is also suitable to build a granular space, becoming the raw material for subsequent feature extraction and recognition of categorized objects.

Single-Exposure HDR Technique Based on Tunable Balance Between Local and Global Adaptation

“…We have used the same prototype image sensor, test board and OpenCV-based software environment as in [9] for the experimental demonstration of the technique just described. All these elements were fully reported in [15]. Each image captured by the sensor is grabbed by the FPGA on the test board and sent to a PC for the on-line adjustment of T adj .…”

Real-time single-exposure ROI-driven HDR adaptation based on focal-plane reconfiguration

“…The scale factor is m = 1 since this chip incorporates additional functionalities at pixel level that require sensing capacitances with the same nominal value. 8 The switches and capacitors are implemented by single MOS transistors. The main characteristics of the chip are summarized in Table 1.…”

Image Feature Extraction Acceleration

“…As an example, we have included the most significant characteristics of our prototypes together with two recently reported focal-plane sensorprocessor chips in Table 1. The Viola-Jones chip embeds extra functionalities in addition to the computation of the integral image [41] while featuring the largest resolution and the smallest pixel pitch, with a cost in terms of a reduced fill factor and increased energy consumption. Concerning the SIFT chip, one of the reasons of the energy overhead is the inherent high number of A/D conversions of the whole Gaussian pyramid plus the input scene, which amounts to 40 A/D conversions of the entire pixel array.…”