An energy-efficient, fast FPGA hardware architecture for OpenCV-Compatible object detection

Abstract

Abstract-The presence of cameras and powerful computers on modern mobile devices gives rise to the hope that they can perform computer vision tasks as we walk around. However, the computational demand and energy consumption of computer vision tasks such as object detection, recognition and tracking make this challenging. At the same time, a fixed vision hard core on the SoC contained in a mobile chip may not have the flexibility needed to adapt to new situations, or evolve as new algorithms are discovered. This may mean that computer vision on a mobile device is the killer application for FPGAs, and could motivate the inclusion of FPGAs, in some form, within modern smartphones. In this paper we present a novel hardware architecture for object detection, that is bit-for-bit compatible with the object classifiers in the widely-used open source OpenCV computer vision software. The architecture is novel, compared to prior work in this area, in two ways: its memory architecture, and its particular SIMD-type of processing. The implementation, which consists of the full system, not simply the kernel, outperforms a same-generation technology mobile processor by a factor of 59 times, and is 13.5 times more energy-efficient.

A multi-processing architecture for accelerating Haar-based face detection on FPGA

“…A maximum clock frequency of 171 MHz was obtained through extensive pipelining and use of fixed point arithmetic instead of floating point. The frame per second (FPS) rate obtained by the proposed architecture, for QVGA resolution, utilizing four processing units and that obtained by utilizing only a single processing unit is compared against the OpenCV software implementation running on an Intel i5, 2.4 GHz processor and the data reported by [8] for a mobile platform. The performance improvement is summarized in Table I.…”

“…[6] accelerated face detection by processing several Haar features within a particular stage in parallel. [8] proposed an architecture where the classifier was run on several different detection windows in parallel. [7] proposed a hybrid solution where the initial stages were heavily parallelized while the later stages were executed sequentially.…”

“…The haar-detection stage lends itself to various forms of parallelizations, some of which have been exploited in the previous works. [6], [7] and [8] have shown that the parallelism that can be extracted from the algorithm is constrained mainly by the amount of resources available on the FPGA. This paper presents a novel architecture that can exploit the inherent parallelism of the Viola-Jones algorithm to significantly accelerate the face-detection process.…”

“…Viola and Jones [1] proposed a machine-learning based approach that can achieve a relatively high frame rate while maintaining accuracy. However, the algorithm is too slow on conventional processors (especially on the embedded platforms) and requires a dedicated co-processor to achieve real-time operation [2], [3], [4], [5], [6], [7], [8].…”

RASW: A run-time adaptive sliding window to improve Viola-Jones object detection

“…RELATED WORK Many hardware implementations of the Viola-Jones face detector have been proposed in recent years to obtain a high throughput [3]- [5]. However, a custom hardware design presents two main drawbacks: i) it requires a significant engineering effort, which also implies high non-recurring costs; ii) it does not provide flexibility, making it almost impossible to adapt the system to any changes in the application scenario.…”