Image Feature Extraction Acceleration

Abstract

Image feature extraction is instrumental for most of the best-performing algorithms in computer vision. However, it is also expensive in terms of computational and memory resources for embedded systems due to the need of dealing with individual pixels at the earliest processing levels. In this regard, conventional system architectures do not take advantage of potential exploitation of parallelism and distributed memory from the very beginning of the processing chain. Raw pixel values provided by the front-end image sensor are squeezed into a high-speed interface with the rest of system components. Only then, after deserializing this massive dataflow, parallelism, if any, is exploited. This chapter introduces a rather different approach from an architectural point of view. We present two Application-Specific Integrated Circuits (ASICs) where the 2-D array of photo-sensitive devices featured by regular imagers is combined with distributed memory supporting concurrent processing. Custom circuitry is added per pixel in order to accelerate image feature extraction right at the focal plane. Specifically, the proposed sensing-processing chips aim at the acceleration of two flagships algorithms within the computer vision community:

A Sub-mW IoT-Endnode for Always-On Visual Monitoring and Smart Triggering

“…To increase the power-efficiency of camera-based architectures, focal plane processing have been investigated [10]. According to this approach, pixel-wise mixed-signal processing circuits are integrated on the sensor die to enable a first stage of processing acceleration in a distributed and efficient way [20]- [22].…”

“…Typically, such devices feature a power consumption of hundreds of mW, because of the high computational and bandwidth requirements [6], [8], [9]. Power-optimized solutions rely on vision chips that integrate focal-plane processing circuits, which enable a first stage of visual processing in a distributed and efficient way at the sensor-level [10]. To extract digital signatures from the visual signal, sensor data are then transferred to the digital processing unit for post-processing operation.…”