Super-Resolution in Plenoptic Cameras Using FPGAs

Abstract

Plenoptic cameras are a new type of sensor that extend the possibilities of current commercial cameras allowing 3D refocusing or the capture of 3D depths. One of the limitations of plenoptic cameras is their limited spatial resolution. In this paper we describe a fast, specialized hardware implementation of a super-resolution algorithm for plenoptic cameras. The algorithm has been designed for field programmable graphic array (FPGA) devices using VHDL (very high speed integrated circuit (VHSIC) hardware description language). With this technology, we obtain an acceleration of several orders of magnitude using its extremely high-performance signal processing capability through parallelism and pipeline architecture. The system has been developed using generics of the VHDL language. This allows a very versatile and parameterizable system. The system user can easily modify parameters such as data width, number of microlenses of the plenoptic camera, their size and shape, and the super-resolution factor. The speed of the algorithm in FPGA has been successfully compared with the execution using a conventional computer for several image sizes and different 3D refocusing planes.

Acceleration techniques and evaluation on multi-core CPU, GPU and FPGA for image processing and super-resolution

“…The proposed 3-pipeline implementation requires similar resources with [19] to achieve one order of magnitude faster execution on the same xc6vlx75t FPGA device. The proposed 11-pipeline implementation on xc6vlx240t consumes a similar number of LUTs with that of the 4-core UHDTV case of [20] to achieve one order of magnitude faster execution than [20] running on Altera Aria II FPGA.…”

“…Pérez et al designed a stream-processing FPGA architecture to super-resolve data from micro-lens arrays in light-field cameras [19]. Their solution requires 105.9 ms to produce 589×589 images from 291×291 micro-lenses on low-power FPGA platforms.…”

Correlation Plenoptic Imaging: An Overview

“…In a more recent scheme, called Plenoptic 2.0, the microlenses create redundant images of portions of the scene of interest, in order to somewhat smoothen the compromise between loss of resolution and increased DOF [16][17][18][19]. Attempts to weaken the resolution vs. DOF trade-off have been made by using signal processing and deconvolution [4,6,[35][36][37], and other algorithms and analysis tools have been developed [8,38].…”

Correlation plenoptic imaging

“…However, the potentials of PI are strongly limited by the inherent inverse proportionality between image resolution and maximum achievable depth of field. Attempts to decouple resolution and depth of field based on signal processing and deconvolution have been proposed in literature [10][11][12][13][14][15].Our idea is to exploit the second-order spatio-temporal correlation properties of light to overcome this fundamental limitation. Using two correlated beams, from either a chaotic or an entangled photon source, we can perform imaging in one arm [16][17][18][19][20][21], and simultaneously obtain the angular information in the other arm.…”

“…However, the potentials of PI are strongly limited by the inherent inverse proportionality between image resolution and maximum achievable depth of field. Attempts to decouple resolution and depth of field based on signal processing and deconvolution have been proposed in literature [10][11][12][13][14][15].…”