Real-Time Refocusing Using an FPGA-Based Standard Plenoptic Camera

Hahne, Christopher; Lumsdaine, Andrew; Aggoun, A.; Velisavljević, Vladan

doi:10.1109/tie.2018.2818644

Cited by 17 publications

(9 citation statements)

References 19 publications

(33 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this configuration, light rays are focused by the MLA on the sensor plane. The calibration of unfocused plenoptic camera [19] has been widely studied in the literature [6,2,25,24,11,36]. Most approaches rely on a thin-lens model for the main lens and an array of pinholes for the micro-lenses.…”

Section: Related Workmentioning

confidence: 99%

Blur Aware Calibration of Multi-Focus Plenoptic Camera

Labussière

Teulière

Bernardin

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

This paper presents a novel calibration algorithm for Multi-Focus Plenoptic Cameras (MFPCs) using only raw images. The design of such cameras is usually complex and relies on precise placement of optic elements. Several calibration procedures have been proposed to retrieve the camera parameters but relying on simplified models, reconstructed images to extract features, or yet multiple calibrations when several types of micro-lens are used. Considering blur information, we propose a new Blur Aware Plenoptic (BAP) feature. It is first exploited in a pre-calibration step that retrieves initial camera parameters, and secondly to express a new cost function for our single optimization process. The effectiveness of our calibration method is validated by quantitative and qualitative experiments.

show abstract

Section: Related Workmentioning

confidence: 99%

Blur Aware Calibration of Multi-Focus Plenoptic Camera

Labussière

Teulière

Bernardin

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…Few researchers proposed FPGA accelerators for plenoptic camera based systems, but they only deal with the 2D refocusing algorithms. Hahne et al [27] used FPGA for the real-time refocusing of standard plenoptic camera. Their results show that FPGA performs better than GPU and general purpose computer with respect to execution time.…”

Section: Related Workmentioning

confidence: 99%

Design of an FPGA Hardware Optimizing the Performance and Power Consumption of a Plenoptic Camera Depth Estimation Algorithm

Bhatti¹,

Greiner²

2021

Algorithms

View full text Add to dashboard Cite

Plenoptic camera based system captures the light-field that can be exploited to estimate the 3D depth of the scene. This process generally consists of a significant number of recurrent operations, and thus requires high computation power. General purpose processor based system, due to its sequential architecture, consequently results in the problem of large execution time. A desktop graphics processing unit (GPU) can be employed to resolve this problem. However, it is an expensive solution with respect to power consumption and therefore cannot be used in mobile applications with low energy requirements. In this paper, we propose a modified plenoptic depth estimation algorithm that works on a single frame recorded by the camera and respective FPGA based hardware design. For this purpose, the algorithm is modified for parallelization and pipelining. In combination with efficient memory access, the results show good performance and lower power consumption compared to other systems.

show abstract

“…In previous work, it has been reported that data-parallel approaches, in which the hyperspectral data is partitioned among different PEs, are particularly effective for parallel processing in the high-performance computing systems like FPGA [40,41]. So, it is crucial to choose a satisfactory strategy for partitioning the HSI data in stage 2.…”

Section: Stage Numbermentioning

confidence: 99%

A Novel FPGA-Based Architecture for Fast Automatic Target Detection in Hyperspectral Images

Lei

et al. 2019

Remote Sensing

View full text Add to dashboard Cite

Onboard target detection of hyperspectral imagery (HSI), considered as a significant remote sensing application, has gained increasing attention in the latest years. It usually requires processing huge volumes of HSI data in real-time under constraints of low computational complexity and high detection accuracy. Automatic target generation process based on an orthogonal subspace projector (ATGP-OSP) is a well-known automatic target detection algorithm, which is widely used owing to its competitive performance. However, ATGP-OSP has an issue to be deployed onboard in real-time target detection due to its iteratively calculating the inversion of growing matrices and increasing matrix multiplications. To resolve this dilemma, we propose a novel fast implementation of ATGP (Fast-ATGP) while maintaining target detection accuracy of ATGP-OSP. Fast-ATGP takes advantage of simple regular matrix add/multiply operations instead of increasingly complicated matrix inversions to update growing orthogonal projection operator matrices. Furthermore, the updated orthogonal projection operator matrix is replaced by a normalized vector to perform the inner-product operations with each pixel for finding a target per iteration. With these two major optimizations, the computational complexity of ATGP-OSP is substantially reduced. What is more, an FPGA-based implementation of the proposed Fast-ATGP using high-level synthesis (HLS) is developed. Specifically, an efficient architecture containing a bunch of pipelines being executed in parallel is further designed and evaluated on a Xilinx XC7VX690T FPGA. The experimental results demonstrate that our proposed FPGA-based Fast-ATGP is able to automatically detect multiple targets on a commonly used dataset (AVIRIS Cuprite Data) at a high-speed rate of 200 MHz with a significant speedup of nearly 34.3 times that of ATGP-OSP, while retaining nearly the same high detection accuracy. data analysis, it is rather rare and may be difficult to find because many of HSI-detected targets are relatively small and weak, such as anomalies [10]. Consequently, real-time target detection in HSI presents a great challenge and has become increasingly important.Currently, many algorithms have been developed for target detection in HSI, such as unsupervised fully constrained least squares (UFCLS) [11], automatic target-generation process based on an orthogonal subspace projector (ATGP-OSP) [12], and Reed-Xiaoli (RX) detector [13]. Among them, the performance of ATGP performs well in terms of detection accuracy and computational complexity according to the quantitative and comparative assessments [14,15]. However, despite the fact that automatic target detection has received increasing attention [16][17][18], two problems with real-time onboard applications need to be addressed. One is to continuously increase the scale of operations in the process of automatically iterating to update currently being found targets, which complicates the process of updating operator matrix. The other one is a large-scale matrix m...

show abstract

Real-Time Refocusing Using an FPGA-Based Standard Plenoptic Camera

Cited by 17 publications

References 19 publications

Blur Aware Calibration of Multi-Focus Plenoptic Camera

Blur Aware Calibration of Multi-Focus Plenoptic Camera

Design of an FPGA Hardware Optimizing the Performance and Power Consumption of a Plenoptic Camera Depth Estimation Algorithm

A Novel FPGA-Based Architecture for Fast Automatic Target Detection in Hyperspectral Images

Contact Info

Product

Resources

About