Fast and Accurate Depth Estimation From Sparse Light Fields

Chuchvara, Aleksandra; Barsi, Attila; Gotchev, Atanas

doi:10.1109/tip.2019.2959233

Cited by 28 publications

(12 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They are then merged together to form a single depth map. In addition to depth accuracy, the authors of [26] set the emphasis on reducing the complexity of the depth estimation algorithm by using superpixels as their basic data units. Using a GPU-optimized implementation, they can process one HD depth map per second.…”

Section: Depth Estimationmentioning

confidence: 99%

Immersive Video Coding: Should Geometry Information Be Transmitted as Depth Maps?

Garus

Henry

Jung

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Immersive video often refers to multiple views with texture and scene geometry information, from which different viewports can be synthesized on the client side. To design efficient immersive video coding solutions, it is desirable to minimize bitrate, pixel rate and complexity. We investigate whether the classical approach of sending the geometry of a scene as depth maps is appropriate to serve this purpose. Previous work shows that bypassing depth transmission entirely and estimating depth at the client side improves the synthesis performance while saving bitrate and pixel rate. In order to understand if the encoder side depth maps contain information that is beneficial to be transmitted, we first explore a hybrid approach which enables partial depth map transmission using a block-based RD-based decision in the depth coding process. This approach reveals that partial depth map transmission may improve the rendering performance but does not present a good compromise in terms of compression efficiency. This led us to address the remaining drawbacks of decoder side depth estimation: complexity and depth map inaccuracy. We propose a novel system that takes advantage of high quality depth maps at the server side by encoding them into lightweight features that support the depth estimator at the client side. These features allow reducing the amount of data that has to be handled during decoder side depth estimation by 88%, which significantly speeds up the cost computation and the energy minimization of the depth estimator. Furthermore, -46.0% and -37.9% average synthesis BD-Rate gains are achieved compared to the classical approach with depth maps estimated at the encoder.

show abstract

Section: Depth Estimationmentioning

confidence: 99%

Immersive Video Coding: Should Geometry Information Be Transmitted as Depth Maps?

Garus

Henry

Jung

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

show abstract

“…As a final step, the depth map is refined using a second CNN, which is designed as an encoder-decoder architecture. Besides of depth accuracy, the authors of [11] set the emphasis on reducing the complexity of the depth estimation algorithm by using superpixels as their basic data units. Using a GPU-optimized implementation, they achieve to estimate around one HD depth map per second.…”

Section: Depth Estimationmentioning

confidence: 99%

Bypassing Depth Maps Transmission For Immersive Video Coding

Garus

Jung

Maugey

et al. 2019

2019 Picture Coding Symposium (PCS)

View full text Add to dashboard Cite

This paper addresses several downsides of the system under development in MPEG-I for coding and transmission of immersive media. We present a solution, which enables Depth-Image-Based Rendering for immersive video applications, while lifting the requirement of transmitting depth information. Instead, we estimate the depth information on the client-side from the transmitted views. The approach leads to an impressive rate saving (37.3% in average). Preserving perceptual quality in terms of MS-SSIM of synthesized views, it yields to 24.6% rate reduction for the same quality of reconstructed views after residue transmission under the MPEG-I common test conditions. Simultaneously, the required pixel rate, i.e. the number of pixels processed per second by the decoder, is reduced by 50% for any test sequence. To the author's knowledge, this is the first time that such an approach is under consideration in the context of immersive video coding.

show abstract

“…The relation between defocus and depth is also exploited by the sub-pixel cost volume of Jeon et al [15], who also present a method for dealing with the distortion induced by micro-lens arrays. An efficient and accurate method for wide-baseline light fields was proposed by Chuchwara et al [7]. They use an oversegmentation of each view to get initial depth proposals, which are iteratively improved using PatchMatch [3].…”

Section: Related Workmentioning

confidence: 99%

Edge-aware Bidirectional Diffusion for Dense Depth Estimation from Light Fields

Khan,

Kim,

Tompkin

2021

Preprint

View full text Add to dashboard Cite

We present an algorithm to estimate fast and accurate depth maps from light fields via a sparse set of depth edges and gradients. Our proposed approach is based around the idea that true depth edges are more sensitive than texture edges to local constraints, and so they can be reliably disambiguated through a bidirectional diffusion process. First, we use epipolar-plane images to estimate sub-pixel disparity at a sparse set of pixels. To find sparse points efficiently, we propose an entropy-based refinement approach to a line estimate from a limited set of oriented filter banks. Next, to estimate the diffusion direction away from sparse points, we optimize constraints at these points via our bidirectional diffusion method. This resolves the ambiguity of which surface the edge belongs to and reliably separates depth from texture edges, allowing us to diffuse the sparse set in a depth-edge and occlusion-aware manner to obtain accurate dense depth maps. Project webpage: http://visual.cs.brown.edu/lightfielddepth

show abstract

Fast and Accurate Depth Estimation From Sparse Light Fields

Cited by 28 publications

References 57 publications

Immersive Video Coding: Should Geometry Information Be Transmitted as Depth Maps?

Immersive Video Coding: Should Geometry Information Be Transmitted as Depth Maps?

Bypassing Depth Maps Transmission For Immersive Video Coding

Edge-aware Bidirectional Diffusion for Dense Depth Estimation from Light Fields

Contact Info

Product

Resources

About