Dataflow-Based Mapping of Computer Vision Algorithms onto FPGAs

Abstract

We develop a design methodology for mapping computer vision algorithms onto an FPGA through the use of coarse-grain reconfigurable dataflow graphs as a representation to guide the designer. We first describe a new dataflow modeling technique called homogeneous parameterized dataflow (HPDF), which effectively captures the structure of an important class of computer vision applications. This form of dynamic dataflow takes advantage of the property that in a large number of image processing applications, data production and consumption rates can vary, but are equal across dataflow graph edges for any particular application iteration. After motivating and defining the HPDF model of computation, we develop an HPDF-based design methodology that offers useful properties in terms of verifying correctness and exposing performance-enhancing transformations; we discuss and address various challenges in efficiently mapping an HPDF-based application representation into target-specific HDL code; and we present experimental results pertaining to the mapping of a gesture recognition application onto the Xilinx Virtex II FPGA.

Discrete event dataflow as a formal approach to specification of industrial vision systems

“…To model typical image processing applications, in HPDF the data production and consumption rate is the same along dataflow graph edges. In [14], the vision algorithms of gesture recognition and face detection were modeled using HPDF, and in [15], the HPDF model was used for mapping a gesture recognition algorithms onto an FPGA board.…”

“…This is the reason why the dataflow research is largely done within the embedded systems community. Though [14] and [15] apply dataflow modeling techniques to computer vision applications, the target use-case remains the design of image processing hardware, and therefore one requires strict formal properties, such as bounded memory requirements and efficient synthesis solutions [15], which restrict the expressiveness of the dataflow models. Dynamic dataflow models therefore seem more suitable for describing more complex data-dependent computer vision applications.…”

Just-in-time scheduling techniques for multicore signal processing systems

“…However, these extensions are based on imperative languages (e.g., C, C++, Fortran) that do not provide mechanisms to model specific signal flow graph topologies. On the contrary, signal processing oriented dataflow MoCs are widely used for specification of data-driven signal flow graphs in a wide range of application areas, including video decoding [13], telecommunication [14], [15], and computer vision [16]. The popularity of dataflow MoCs in design and implementation of signal processing systems is due largely to their analyzability and their natural expressivity of the concurrency in signal processing algorithms, which makes them suitable for exploiting the parallelism offered by MPSoCs.…”

Implementation of Wireless Vision Sensor Node for Characterization of Particles in Fluids

“…The advantages and disadvantages of FPGA technology and its suitability for computer vision tasks were discussed in detail in [10] and its optimization in [11]. A design methodology for mapping the computer vision algorithm onto an FPGA through the use of a coarse grain reconfigurable data flow graph was discussed in detail in [12].…”