In‐pixel analog memories for a pixel‐based background subtraction algorithm on CMOS vision sensors

Abstract

This paper addresses the most suitable in-pixel analog memory bank for running a hardware-oriented approach of the well-known Pixel-Based Adaptive Segmenter algorithm on CMOS vision sensors. The high number of memory accesses, typically up to 200 times, along with the long time elapsed before an analog memory in a pixel is updated, around 5 seconds at 30 fps, constrains the memory topology. This work assesses the impact of nominal and process nonidealities of the 3 main analog memory topologies, namely, open-loop, closed-loop, and integrator architectures for background subtraction over the CDNET14 database. This is the first step towards the implementation of a CMOS vision chip with per-pixel processing to run the Pixel-Based Adaptive Segmenter. KEYWORDS analog memories, background subtraction, CMOS vision sensors, focal plane processing, PBAS *Corrections added on 25 April 2018, after first online publication: several incorrect citations have been corrected and a missing reference [19] has been added.

Deep Learning-Based Multiple Object Visual Tracking on Embedded System for IoT and Mobile Edge Computing Applications

“…The Hardware-Oriented PBAS (HO-PBAS) algorithm [16] to detect moving objects was integrated with the GOTURN CNN based tracker [17] in order to perform multiple object tracking. Whereas both algorithms present good results separately, additional development was needed to overcome some limitations due to the jointly operation.…”

“…A proposal for multiple object tracking was developed making use of the Hardware-Oriented PBAS (HO-PBAS) algorithm [16] to detect moving objects integrated with the GO-TURN CNN based tracker [17]. The HO-PBAS detector [16] is a hardware oriented foreground segmentation method based on the pixel-based adaptive segmenter (PBAS) algorithm [37].…”

“…A proposal for multiple object tracking was developed making use of the Hardware-Oriented PBAS (HO-PBAS) algorithm [16] to detect moving objects integrated with the GO-TURN CNN based tracker [17]. The HO-PBAS detector [16] is a hardware oriented foreground segmentation method based on the pixel-based adaptive segmenter (PBAS) algorithm [37]. It is oriented to focal-plane processing, [38], with the benefit of less memory usage, becoming appropriate for future onchip implementation of the whole algorithm and also for its use on embedded solutions such as the Jetson platform.…”

“…Next, our own implementation of the GOTURN CNN tracker as a multiple object tracking proposal is described in Section III. A Hardware-Oriented PBAS (HO-PBAS) algorithm [16] is used to detect moving objects and it is integrated with the GOTURN CNN based tracker [17]. Moreover, additional code was included to manage multiple object tracking.…”

Guest editorial: Special issue on computational image sensors and smart camera hardware

“…Finally, the article “In‐Pixel Analog Memories for a Pixel‐Based Background Subtraction Algorithm on CMOS Vision Sensors” by García‐Lesta et al describes a focal‐plane sensing‐processing array to accelerate image segmentation. The authors propose a hardware‐oriented implementation featuring less memory per‐pixel than commonly required.…”