Adaptive Subsampling for ROI-Based Visual Tracking: Algorithms and FPGA Implementation

Iqbal, Odrika; Muro, Victor Isaac Torres; Katoch, Sameeksha; Spanias, Andreas; Jayasuriya, Suren

doi:10.1109/access.2022.3200755

Cited by 7 publications

(18 citation statements)

References 84 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We employ an LSTM network in lieu of the Kalman filter to make our future location predictions. An extension of the work [60] also showed how the ECO tracker coupled with the Kalman filter could be leveraged for greater ROI prediction accuracy. However, the LSTM network benefits from past temporal information encoded in its hidden units and is able to anticipate future trajectories with a great degree of accuracy than the ECO tracker plus Kalman filter-based predictive algorithm (see Section V).…”

Section: Related Workmentioning

confidence: 97%

“…Brief Methodology Highlights Limitations YOLO [14] +KF The convolutional neural network (YOLO) extracts image features and the Kalman filter predicts ROIbased sensor masks as demonstrated in [60].…”

Section: Algorithmmentioning

confidence: 99%

“…The KCF demonstrates performance degradation when it comes to occluded and out-of-view objects and does not have a built-in selfcorrection mechanism in the tracking process. DAT [61] ECO+KF [60] Efficient Convolution Operators for Tracking (ECO) is leveraged as a feature detector in this method with the Kalman filter (KF) acting as the ROI predictor as shown in [60]. The ECO tracker features a factorized convolution operator for parameter reduction, a generative model for improved characterization of the input data, and a model update strategy that updates the model at certain intervals.…”

Section: Algorithmmentioning

confidence: 99%

See 2 more Smart Citations

Energy-Efficient Object Tracking Using Adaptive ROI Subsampling and Deep Reinforcement Learning

et al. 2023

View full text Add to dashboard Cite

Recent innovations in ROI camera systems have opened up the avenue for exploring energy optimization techniques like adaptive subsampling. Generally speaking, image frame capture and read-out demand high power consumption. ROI camera systems make it possible to exploit the inverse relation between energy consumption and spatiotemporal pixel readout to optimize the power efficiency of the image sensor. To this end, we develop a reinforcement learning (RL) based adaptive subsampling framework which predicts ROI trajectories and reconfigures the image sensor on-the-fly for improved power efficiency of the image sensing pipeline. In our proposed framework, a pre-trained convolutional neural network (CNN) extracts rich visual features from incoming frames and a long short-term memory (LSTM) network predicts the region of interest (ROI) and subsampling pattern for the consecutive image frame. Based on the application and the difficulty level of object motion trajectory, the user can utilize either the predicted ROI or coarse subsampling pattern to switch off the pixels for sequential frame capture, thus saving energy. We have validated our proposed method by adapting existing trackers for the adaptive subsampling framework and evaluating them as competing baselines. As a proof-of-concept, our method outperforms the baselines and achieves an average AUC score of 0.5090 on three benchmarking datasets. We also characterize the energy-accuracy tradeoff of our method vs. the baselines and show that our approach is best suited for applications that demand both high visual tracking precision and low power consumption. On the TB100 dataset, our method achieves the highest AUC score of 0.5113 out of all the competing algorithms and requires a medium-level power consumption of approximately 4 W as per a generic energy model and an energy consumption of 1.9 mJ as per a mobile system energy model. Although other baselines are shown to have better performance in terms of power consumption, they are ill-suited for applications that require considerable tracking precision, making our method the ideal candidate in terms of power-accuracy tradeoff.

show abstract

Section: Related Workmentioning

confidence: 97%

Section: Algorithmmentioning

confidence: 99%

Section: Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

Energy-Efficient Object Tracking Using Adaptive ROI Subsampling and Deep Reinforcement Learning

et al. 2023

View full text Add to dashboard Cite

show abstract

“…It is essential to make a tradeoff between the added energy cost of the data reduction technique, the final gain in energy from its implementation, and the QoS degradation. Many approaches have been proposed in this context to achieve the intended target by using low-cost and classical techniques or advanced techniques based on machine learning (ML) and deep learning (DL) [34].…”

Section: Background and Related Workmentioning

confidence: 99%

An Efficient Low Complexity Region-of-Interest Detection for Video Coding in Wireless Visual Surveillance

et al. 2023

View full text Add to dashboard Cite

Following its use in several applications, including video coding in wireless surveillance, moving object detection (MOD) has become a popular video analysis topic. Despite the considerable progress in the accuracy of MOD for video coding, its implementation in constrained sensors is a real challenge owing to their high complexity and energy consumption. Therefore, there is a great need to address the trade-off between the accuracy and the energy efficiency of MOD approaches for video coding in constrained systems. In this work, an energy-efficient region-of-interest (ROI) detection algorithm as a pre-encoder for wireless visual surveillance (WVS) is proposed. The algorithm ensures a trade-off between detection accuracy and computational complexity. To this end, we propose constructing an activity map by measuring each block activity between successive frames. The map scores are processed using a combination of a fast Gaussian smoother and a rank-order filter to improve accuracy. Only the blocks in motion are coded and counted for transmission. The accuracy of our approach has been evaluated on a large dataset using key performance metrics. It has been found that our algorithm outperforms other state-of-theart techniques in terms of true positive rate (TPR), with 80.84% on sensitivity metric, while exhibiting a well-balanced accuracy for all categories. A careful examination of the computational complexity confirms the low overhead. The energy and bitrate savings could achieve nearly 90% and 98%, respectively.

show abstract

“…Two core computer vision algorithm functions need to be used in this process, which are "Crop" and "Sum". The crop function is used to extract the region of interest (ROI) [28] from the input image and the sum function is used to calculate the sum of all pixels in the input image.…”

Section: Memory-tree Algorithm In Ocr Systemmentioning

confidence: 99%

Memory-Tree Based Design of Optical Character Recognition in FPGA

Kim

Choi

2023

Electronics

View full text Add to dashboard Cite

As one of the fields of Artificial Intelligence (AI), Optical Character Recognition (OCR) systems have wide application in both industrial production and daily life. Conventional OCR systems are commonly designed and implement data computation on the basis of microprocessors; the performance of the processor relates to the effect of the computation. However, due to the “Memory-wall” problem and Von Neumann bottlenecks, the drawbacks of traditional processor-based computing for OCR systems are gradually becoming apparent. In this paper, an approach based on the Memory-Centric Computing and “Memory-Tree” algorithm has been proposed to perform hardware optimization of traditional OCR systems. The proposed algorithm was first designed in software implementation using C/C++ and OpenCV to verify the feasibility of the idea and then the RTL conversion of the algorithm was done using the Xilinx Vitis High Level Synthesis (HLS) tool to implement the hardware. This work chose Xilinx Alveo U50 FPGA Accelerator to complete the hardware design, which can be connected to the x86 CPU in the PC by PCIe to form heterogeneous computing. The results of the hardware implementation show that the system this work designed can recognize characters of English capital letters and numbers within 34.24 us. The power of FPGA is 18.59 W, which saves 77.87% of energy consumption compared to the 84 W of the processor in PC.

show abstract

Adaptive Subsampling for ROI-Based Visual Tracking: Algorithms and FPGA Implementation

Cited by 7 publications

References 84 publications

Energy-Efficient Object Tracking Using Adaptive ROI Subsampling and Deep Reinforcement Learning

Energy-Efficient Object Tracking Using Adaptive ROI Subsampling and Deep Reinforcement Learning

An Efficient Low Complexity Region-of-Interest Detection for Video Coding in Wireless Visual Surveillance

Memory-Tree Based Design of Optical Character Recognition in FPGA

Contact Info

Product

Resources

About