Custom processor design for efficient, yet flexible Lucas-Kanade optical flow

Abstract

State-of-the-art solutions to optical flow fail to jointly offer high density flow estimation, low power consumption and real time operation, rendering them unsuitable for embedded applications. Joint hardware-software scalability at run-time is crucial to achieve these conflicting requirements in one device. This paper therefore presents a scalable Lucas-Kanade optical flow algorithm, together with a flexible power-optimized processor architecture. The C-programmable processor exploits algorithmic scalability through innovations in its memory structure, memory interface, and datapath optimized for efficient convolutions. Jointly, the scalable flow algorithm and optimized computer vision hardware platform enable applications to on-the-fly tradeoff throughput and power consumption in function of flow density and accuracy. The processor chip is synthesized in 40nm CMOS technology and verified on FPGA. The architecture is capable of scaling the frame rate at run-time and processes 16fps of dense optical flow at 640×480 resolution with 15.06 • average angular error, while only consuming 24mW.

Real-Time Efficient FPGA Implementation of the Multi-Scale Lucas-Kanade and Horn-Schunck Optical Flow Algorithms for a 4K Video Stream

“…Another work is [29], in which the authors proposed adaptive multi-scale processor able to process the LK algorithm with different density, precision, energy consumption and number of scales (up to 4). The architecture was verified on the Nexys 4 FPGA board, and the processor itself can process 640 × 480 (VGA) @ 16 fps video stream, while using only 24 mW of energy.…”

Motion compensation for the estimation of high-resolution blood flow in ultrafast ultrasound imaging

“…We compare this method to techniques folding in the first strategy, namely an explicit motion estimation prior to the blood separation. For this purpose, we consider a motion estimation based on an optical flow approach, namely the Pyramidal Lucas-Kanade algorithm [8]. Finally, we also compare these approaches to the algorithm proposed in [6], which includes a background-foreground separation of the estimated motion using a stable principal component pursuit (SPCP) algorithm.…”