A Computationally Efficient Visual Saliency Algorithm Suitable for an Analog CMOS Implementation

Abstract

Computer vision algorithms are often limited in their application by the large amount of data that must be processed. Mammalian vision systems mitigate this high bandwidth requirement by prioritizing certain regions of the visual field with neural circuits that select the most salient regions. This work introduces a novel and computationally efficient visual saliency algorithm for performing this neuromorphic attention-based data reduction. The proposed algorithm has the added advantage that it is compatible with an analog CMOS design while still achieving comparable performance to existing state-of-the-art saliency algorithms. This compatibility allows for direct integration with the analog-to-digital conversion circuitry present in CMOS image sensors. This integration leads to power savings in the converter by quantizing only the salient pixels. Further system-level power savings are gained by reducing the amount of data that must be transmitted and processed in the digital domain. The analog CMOS compatible formulation relies on a pulse width (i.e., time mode) encoding of the pixel data that is compatible with pulse-mode imagers and slope based converters often used in imager designs. This letter begins by discussing this time-mode encoding for implementing neuromorphic architectures. Next, the proposed algorithm is derived. Hardware-oriented optimizations and modifications to this algorithm are proposed and discussed. Next, a metric for quantifying saliency accuracy is proposed, and simulation results of this metric are presented. Finally, an analog synthesis approach for a time-mode architecture is outlined, and postsynthesis transistor-level simulations that demonstrate functionality of an implementation in a modern CMOS process are discussed.

Perovskite-Based Memristor with 50-Fold Switchable Photosensitivity for In-Sensor Computing Neural Network

“…The traditional artificial visual system depends on the CMOS-based image sensor and a control system to convert the optical signal into voltage spikes [45][46][47]. Then, the data are transmitted to the computer for neural network function calculation.…”

“…To exclude the influence of bias voltage, the readout voltage is uniformly set to 0.2 V. It can be seen from Figure 3g-i that, in a low-light environment, the higher the intrinsic photoresponsivity of the device, the easier it is to obtain more explicit images and more details. The traditional artificial visual system depends on the CMOS-based image sensor and a control system to convert the optical signal into voltage spikes [45][46][47]. Then, the data are transmitted to the computer for neural network function calculation.…”