FPGA Hardware Design for Plenoptic 3D Image Processing Algorithm Targeting a Mobile Application

Abstract

Over the past years, widespread use of applications based on 3D image processing has increased rapidly. It is being employed in various fields, such as research, medicine and automation. Plenoptic camera system is used to capture lightfield that can be exploited to estimate the 3D depth of the scene. The respective algorithms consist of a large number of computation-intensive instructions. It eventually leads to the problem of large execution time of the algorithm. Moreover, they require substantial amount of memory cells for the storage of intermediate and final results. Desktop GPU based solutions are power intensive and therefore cannot be used in the mobile applications with low energy requirements. The idea presented in this paper is to use the FPGA based hardware design to improve the performance of a 3D depth estimation algorithm by utilizing the advantage of concurrent execution. The algorithm is implemented, evaluated and the results show that FPGA design reduces the respective execution time significantly.

FPGA Implementation of CNN Based DepthEstimation Network: MiDaSNet

“…[9] implemented a depth-from-motion model using an optical flow algorithm on FPGA, which offers high speed and low resource consumption on hardware; however, the results are motion-dependent and show lower accuracy compared to more sophisticated NN-based methods. [10] proposed FPGA implementation of a plenoptic depth estimation method which uses stereo-matching with recursive census transform to calculate the sparse depth from micro-lens-arrays. Although it results in fast and accurate depth estimation with relatively lower computational power consumption, the depth map calculated from MLAs is sparse and an increase in pixel density, would result in excessive resource consumption and higher delays.…”