Integral Images: Efficient Algorithms for Their Computation and Storage in Resource-Constrained Embedded Vision Systems

Abstract

The integral image, an intermediate image representation, has found extensive use in multi-scale local feature detection algorithms, such as Speeded-Up Robust Features (SURF), allowing fast computation of rectangular features at constant speed, independent of filter size. For resource-constrained real-time embedded vision systems, computation and storage of integral image presents several design challenges due to strict timing and hardware limitations. Although calculation of the integral image only consists of simple addition operations, the total number of operations is large owing to the generally large size of image data. Recursive equations allow substantial decrease in the number of operations but require calculation in a serial fashion. This paper presents two new hardware algorithms that are based on the decomposition of these recursive equations, allowing calculation of up to four integral image values in a row-parallel way without significantly increasing the number of operations. An efficient design strategy is also proposed for a parallel integral image computation unit to reduce the size of the required internal memory (nearly 35% for common HD video). Addressing the storage problem of integral image in embedded vision systems, the paper presents two algorithms which allow substantial decrease (at least 44.44%) in the memory requirements. Finally, the paper provides a case study that highlights the utility of the proposed architectures in embedded vision systems.

FPGA-based Binocular Image Feature Extraction and Matching System

“…Image integration, the first step in SURF detection, is implemented by the Integrator module. For details of the Integrator, please refer to [16]. Figure 3).…”

Ground Control Point Automatic Extraction for Spaceborne Georeferencing Based on FPGA

“…Using this method, the integral image memory can be reduced to 42.6% for a 640 × 480 8-bit grayscale image. Ehsan et al [64]- [66] proposed a parallel recursive equation to compute the integral image. This method not only substantially decreases the operation and memory requirements (by at least 44.44%) but also maintains the accuracy.…”

“…The total memory in bytes required to store the integral image is (W × H) × L ii /8. In general, the maximum width and height of the box filter are known, and the word length for the integral image using the exact method [66] with complement-coded arithmetic needs to satisfy…”

“…However, the Viola-Jones method has time delay drawbacks. To accelerate the processing of an integral image, Ehsan et al [66] proposed an n stage of a pipelined system that processes n rows of an input image in parallel, providing n integral image value per clock cycles without delay when the pipeline is full. This method can be mathematically defined as follows:…”

“…., n/2 − 1. This set of equations requires 2MN + MN/2 addition operation for an input image with the resolution M × N pixels [66]. Compared with Viola-Jones equation, the increase is not significant.…”