VLSI Architecture Study of a Real-Time Scalable Optical Flow Processor for Video Segmentation

Abstract

An optical flow processor architecture is proposed. It offers accuracy and image-size scalability for video segmentation extraction. The Hierarchical Optical flow Estimation (HOE) algorithm [1] is optimized to provide an appropriate bit-length and iteration number to realize VLSI. The proposed processor architecture provides the following features. First, an algorithm-oriented data-path is introduced to execute all necessary processes of optical flow derivation allowing hardware cost minimization. The data-path is designed using 4-SIMD architecture, which enables highthroughput operation. Thereby, it achieves real-time optical flow derivation with 100% pixel density. Second, it has scalable architecture for higher accuracy and higher resolution. A third feature is the CMOS-process compatible on-chip 2-port DRAM for die-area reduction. The proposed processor has performance for CIF 30 fr/s with 189 MHz clock frequency. Its estimated core size is 6.02 × 5.33 mm 2 with six-metal 90-nm CMOS technology. key words: optical flow, processor architecture, video segmentation Kousuke Imamura received B.S., M.S. and Dr. Eng. degrees in Electrical Engineering and , he was engaged in the design of NMOS and CMOS static RAM, including a 64-Kb full CMOS RAM with a divided-word-line structure. Since 1984 he has been involved in research and development of multimedia ULSI systems for digital broadcasting and digital communication systems based on MPEG-2 and MPEG-4 Codec LSI core technology. Since 2000, he had been a Professor of the

A Design for the 178-MHz WXGA 30-fps Optical Flow Processor Based on the HOE Algorithm

“…Finally, our calculations of the flow, when using the 4 diagonal pixels as the local average and applying the GaussSeidel and SOR methods, are given in (8). With these techniques, the computational complexity associated with the iterations was reduced to 5% when compared to the previous HOE processor [6] while retaining the same accuracy. Other computations, such as generation of hierarchical images and motion compensation frames, account for 10% of the total.…”

“…The HOE [6] algorithm, which is based on the HornSchunck algorithm, gives a highly accurate optical flow using a motion compensation frame. The luminance value of the pixels ( , ) in the time frame is ( , , ), and each pixel moves and respectively, in the direction and in the next frame at the time + 1.…”

“…When the updated value of the flow in the iteration becomes smaller than the preset threshold, or when the number of iterations reaches the preset number, the calculation ends. In [6], the updated threshold is set to 0.0001 and the number of iterations to 150 times. The flow obtained from this hierarchy is doubled and then propagated to the lower layer.…”

“…Hereafter, this method is called the image division method. The capacity of the embedded memory is reduced to 1/N compared to the previous HOE processor [6], where N is the number of blocks. If the frame is simply divided into blocks, great flow disturbance occurs in the block boundary, which degrades the accuracy of the flow.…”

“…1 (b), the vertical axis also shows the resolution while the horizontal axis shows the operating frequency. The Hierarchical Optical flow Estimation processor (HOE) [6] based on the Horn-Shunck algorithm, and the Pyramidal Lucas-Kanade processor (PLK) [7] based on the Lucas-Kanade algorithm, are highly accurate. However, as shown in Fig.…”