A Taxonomy and Evaluation of Dense Light Field Depth Estimation Algorithms

Johannsen, Ole; Honauer, Katrin; Goldluecke, Bastian; Alperovich, Anna; Battisti, Federica; Bok, Yunsu; Brizzi, Michele; Carli, Marco; Choe, Gyeongmin; Diebold, Maximilian; Gutsche, Marcel; Jeon, Hae‐Gon; Kweon, In So; Park, Jaesik; Park, Jinsun; Schilling, Hendrik; Sheng, Hao; Si, Lipeng; Strecke, Michael; Sulc, Antonin; Tai, Yu-Wing; Wang, Qing; Wang, Ting-Chun; Wanner, Sven; Zhang, Xiong; Yu, Jingyi; Zhang, Shuo; Zhu, Hao

doi:10.1109/cvprw.2017.226

Cited by 66 publications

(34 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The disparity range is discretized for the methods [2], [4], [5]. As suggested in the light field depth estimation challenge held in 2017 LF4CV workshop [21], the number of disparity levels is set to 100 for the method [4] and 256 for the method [2]. For the method [5], the disparity step is set to 0.01, which corresponds to the minimal threshold of bad pixel ratios that we use.…”

Section: Results With Densely Sampled Synthetic Light Fieldsmentioning

confidence: 99%

See 1 more Smart Citation

A Framework for Learning Depth From a Flexible Subset of Dense and Sparse Light Field Views

Shi

Jiang

Guillemot

2019

IEEE Trans. on Image Process.

116

View full text Add to dashboard Cite

Light field is a type of image data that captures the 3D scene information by recording light rays emitted from a scene at various orientations. It offers a more immersive perception than classic 2D images but at the cost of huge data volume. In this paper, we draw inspiration from the visual characteristics of Sub-Aperture Images (SAIs) of light field and design a compact neural network representation for the light field compression task. The network backbone takes randomly initialized noise as input and is supervised on the SAIs of the target light field. It is composed of two types of complementary kernels: descriptive kernels (descriptors) that store scene description information learned during training, and modulatory kernels (modulators) that control the rendering of different SAIs from the queried perspectives. To further enhance compactness of the network meanwhile retain high quality of the decoded light field, we accordingly introduce modulator allocation and kernel tensor decomposition mechanisms, followed by non-uniform quantization and lossless entropy coding techniques, to finally form an efficient compression pipeline. Extensive experiments demonstrate that our method outperforms other state-of-the-art (SOTA) methods by a significant margin in the light field compression task. Moreover, after aligning descriptors, the modulators learned from one light field can be transferred to new light fields for rendering dense views, indicating a potential solution for view synthesis task.

show abstract

Section: Results With Densely Sampled Synthetic Light Fieldsmentioning

confidence: 99%

“…According to the metrics defined in [20] [21], experimental results show that the proposed approach outperforms state-ofthe-art light field disparity estimation methods for both densely and sparsely sampled LF. In addition, it does not require any prior information on disparity range as in [2], [4], [5] for example.…”

Section: Introductionmentioning

confidence: 99%

A Framework for Learning Depth From a Flexible Subset of Dense and Sparse Light Field Views

Shi

Jiang

Guillemot

2019

IEEE Trans. on Image Process.

116

View full text Add to dashboard Cite

show abstract

“…Ground truth disparity on center view is available for HCI synthetic light fields. Using the evaluation metrics defined in [10,11], we compare in Table 1 our scheme against two state-of-the-art methods, namely *LF [2] and SPO [5]. In our experiments, the number of discretized disparity levels is kept the same as in [11], i.e.…”

Section: Performance Assessmentmentioning

confidence: 99%

“…Unlike [2,5,7,8], the proposed method does not demand discretization of the disparity space, nor prior knowledge about the disparity range. According to the metrics defined in [10,11], the experiments show that our approach achieves competitive performance compared to state-of-the-art methods that make use of the whole set of light field views.…”

Section: Introductionmentioning

confidence: 99%

Depth Estimation with Occlusion Handling from a Sparse Set of Light Field Views

Jiang¹,

Pendu²,

Guillemot³

2018

2018 25th IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

This paper addresses the problem of depth estimation for every viewpoint of a dense light field, exploiting information from only a sparse set of views. This problem is particularly relevant for applications such as light field reconstruction from a subset of views, for view synthesis and for compression. Unlike most existing methods for scene depth estimation from light fields, the proposed algorithm computes disparity (or equivalently depth) for every viewpoint taking into account occlusions. In addition, it preserves the continuity of the depth space and does not require prior knowledge on the depth range. The experiments show that, both for synthetic and real light fields, our algorithm achieves competitive performance to state-of-the-art algorithms which exploit the entire light field and usually generate the depth map for the center viewpoint only.

show abstract

“…3: Visual comparison of the estimated disparity maps on center view. We use the same evaluation metrics defined in [21,22]. MSE is mean-square-error, which penalizes large disparity errors on the object boundary, whereas BadPix(α) (the percentage of pixels having an error superior to α, α being set to small values) and Q25 (the error value *100 at the 25th percentile of the disparity estimates) measure the sub-pixel accuracy.…”

Section: Dense Light Fieldsmentioning

confidence: 99%

A Learning Based Depth Estimation Framework for 4D Densely and Sparsely Sampled Light Fields

Jiang

Shi

Guillemot

2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

This paper proposes a learning based solution to disparity (depth) estimation for either densely or sparsely sampled light fields. Disparity between stereo pairs among a sparse subset of anchor views is first estimated by a fine-tuned FlowNet 2.0 network adapted to disparity prediction task. These coarse estimates are fused by exploiting the photo-consistency warping error, and refined by a Multi-view Stereo Refinement Network (MSRNet). The propagation of disparity from anchor viewpoints towards other viewpoints is performed by an occlusion-aware soft 3D reconstruction method. The experiments show that, both for dense and sparse light fields, our algorithm outperforms significantly the state-of-the-art algorithms, especially for subpixel accuracy.

show abstract

A Taxonomy and Evaluation of Dense Light Field Depth Estimation Algorithms

Cited by 66 publications

References 20 publications

A Framework for Learning Depth From a Flexible Subset of Dense and Sparse Light Field Views

A Framework for Learning Depth From a Flexible Subset of Dense and Sparse Light Field Views

Depth Estimation with Occlusion Handling from a Sparse Set of Light Field Views

A Learning Based Depth Estimation Framework for 4D Densely and Sparsely Sampled Light Fields

Contact Info

Product

Resources

About