FPGA-Based Detection and Tracking System for Surveillance Camera

Mary, Anitha; Rose, Lina; Karunakaran, Aldrin

doi:10.1016/b978-0-12-816385-6.00012-x

Cited by 2 publications

(1 citation statement)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most algorithms that operate online using data from a single image detect defective pixels by computing statistical variations within a neighborhood centered around each pixel. Although these algorithms frequently run on programmable architectures such as traditional computers and embedded processors, custom hardware architectures can provide higher performance with lower resources and power consumption, which are required for applications such as assisted-driving automobiles [ 20 , 21 , 22 ], surveillance systems [ 23 , 24 , 25 ], and biometric recognition [ 26 , 27 , 28 ], among many others. Numerous researchers have proposed custom hardware devices to perform image processing in real time, which reduces power consumption and increases hardware integration.…”

Section: Introductionmentioning

confidence: 99%

A CMOS Image Readout Circuit with On-Chip Defective Pixel Detection and Correction

López-Portilla

Valenzuela

Zarkesh-Ha

et al. 2023

Sensors

View full text Add to dashboard Cite

Images produced by CMOS sensors may contain defective pixels due to noise, manufacturing errors, or device malfunction, which must be detected and corrected at early processing stages in order to produce images that are useful to human users and image-processing or machine-vision algorithms. This paper proposes a defective pixel detection and correction algorithm and its implementation using CMOS analog circuits, which are integrated with the image sensor at the pixel and column levels. During photocurrent integration, the circuit detects defective values in parallel at each pixel using simple arithmetic operations within a neighborhood. At the image-column level, the circuit replaces the defective pixels with the median value of their neighborhood. To validate our approach, we designed a 128×128-pixel imager in a 0.35μm CMOS process, which integrates our defective-pixel detection/correction circuits and processes images at 694 frames per second, according to post-layout simulations. Operating at that frame rate, our proposed algorithm and its CMOS implementation produce better results than current state-of-the-art algorithms: it achieves a Peak Signal to Noise Ratio (PSNR) and Image Enhancement Factor (IEF) of 45 dB and 198.4, respectively, in images with 0.5% random defective pixels, and a PSNR of 44.4 dB and IEF of 194.2, respectively, in images with 1.0% random defective pixels.

show abstract

Section: Introductionmentioning

confidence: 99%

A CMOS Image Readout Circuit with On-Chip Defective Pixel Detection and Correction

López-Portilla

Valenzuela

Zarkesh-Ha

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

Adaptive Subsampling for ROI-Based Visual Tracking: Algorithms and FPGA Implementation

et al. 2022

View full text Add to dashboard Cite

There is tremendous scope for improving the energy efficiency of embedded vision systems by incorporating programmable region-of-interest (ROI) readout in the image sensor design. In this work, we study how ROI programmability can be leveraged for vision applications by anticipating where the ROI will be located in future frames and switching pixels off outside of this region. We refer to this process of ROI prediction and corresponding sensor configuration as adaptive subsampling. Our adaptive subsampling algorithms comprise an object detector and an ROI predictor (Kalman filter) which operate in conjunction to optimize the energy efficiency of the vision pipeline with the end task being object tracking. To further facilitate the implementation of our adaptive algorithms in real systems, we select a candidate algorithm and map it onto an FPGA. Leveraging Xilinx Vitis AI tools, we designed and accelerated a YOLO object detector-based adaptive subsampling algorithm. In order to further improve the algorithm post-deployment, we evaluated several competing baselines on the OTB100 and LaSOT datasets. We found that coupling the ECO tracker with the Kalman filter has a competitive AUC score of 0.4568 and 0.3471 on the OTB100 and LaSOT datasets respectively. Further, the power efficiency of this algorithm is on par with, and in a couple of instances superior to, the other baselines. The ECO-based algorithm incurs a power consumption of approximately 4 W averaged across both datasets while the YOLO-based approach requires power consumption of approximately 6 W (as per our power consumption model). In terms of accuracy-latency tradeoff, the ECO-based algorithm provides near-real-time performance (19.23 FPS) while managing to attain competitive tracking precision.INDEX TERMS Adaptive subsampling, embedded computer vision, FPGA acceleration, hardware/software co-design, single object tracking, vision applications

show abstract

FPGA-Based Detection and Tracking System for Surveillance Camera

Cited by 2 publications

References 13 publications

A CMOS Image Readout Circuit with On-Chip Defective Pixel Detection and Correction

A CMOS Image Readout Circuit with On-Chip Defective Pixel Detection and Correction

Adaptive Subsampling for ROI-Based Visual Tracking: Algorithms and FPGA Implementation

Contact Info

Product

Resources

About