MODE: Multi-view Omnidirectional Depth Estimation with 360$$^\circ $$ Cameras

Li, Ming; Jin, Xueqian; Hu, Xuejiao; Dai, Jingzhao; Du, Sidan; Li, Yang

doi:10.1007/978-3-031-19827-4_12

Cited by 5 publications

(6 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CSDNet [25] presents an approach for panoramic depth estimation using spherical convolutions and epipolar constraints. MODE [7] uses Cassini [26] projection for spherical stereo matching.…”

Section: Omnidirectional Depth Estimationmentioning

confidence: 99%

“…. , S L , we can obtain the ERP features at each index based on Equations ( 4), ( 6) and (7). By concatenating these features together, we form the spherical feature volume for the i th camera, which is denoted as…”

Section: Spherical Cost Volumementioning

confidence: 99%

“…While simplifying system hardware requirements, the limited input information risks inaccurate results in certain contexts, particularly those involving distant or heavily occluded objects. Other methods [5][6][7] incorporate conventional binocular stereo matching constraints and epipolar geometry into panoramic image pairs, predicting more reliable depth. Nevertheless, the practical applicability remains limited due to challenges in obtaining such panoramas and additional calibration requirements.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

CasOmniMVS: Cascade Omnidirectional Depth Estimation with Dynamic Spherical Sweeping

Wang,

Li,

Cao

et al. 2024

Applied Sciences

Self Cite

View full text Add to dashboard Cite

Estimating 360∘ depth from multiple cameras has been a challenging problem. However, existing methods often adopt a fixed-step spherical sweeping approach with densely sampled spheres and use numerous 3D convolutions in networks, which limits the speed of algorithms in practice. Additionally, obtaining high-precision depth maps of real scenes poses a challenge for the existing algorithms. In this paper, we design a cascade architecture using a dynamic spherical sweeping method that progressively refines the depth estimation from coarse to fine over multiple stages. The proposed method adaptively adjusts sweeping intervals and ranges based on the predicted depth and the uncertainty from the previous stage, resulting in a more efficient cost aggregation performance. The experimental results demonstrated that our method achieved state-of-the-art accuracy with reduced GPU memory usage and time consumption compared to the other methods. Furthermore, we illustrate that our method achieved satisfactory performance on real-world data, despite being trained on synthetic data, indicating its generalization potential and practical applicability.

show abstract

“…CSDNet [25] presents an approach for panoramic depth estimation using spherical convolutions and epipolar constraints. MODE [7] uses Cassini [26] projection for spherical stereo matching.…”

Section: Omnidirectional Depth Estimationmentioning

confidence: 99%

Section: Spherical Cost Volumementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

CasOmniMVS: Cascade Omnidirectional Depth Estimation with Dynamic Spherical Sweeping

Wang,

Li,

Cao

et al. 2024

Applied Sciences

Self Cite

View full text Add to dashboard Cite

show abstract

“…The authors proposed a model combining geometric structure estimation with depth estimation based on the prior that geometric structure is associated with depth information [17]. Li et al [18] proposed a multi-view panoramic depth estimation network, which first estimates depth maps from different camera pairs by stereo matching and then fuses the depth maps to achieve robustness to mud spots, water droplets on camera lenses, and glare caused by bright light. Jin et al [17] put forward a multitask model combining geometric structure estimation and depth estimation from the prior that geometric structure is associated with depth information, and both can assist each other in optimization.…”

Section: Related Workmentioning

confidence: 99%

“…Li et al. [18] proposed a multi‐view panoramic depth estimation network, which first estimates depth maps from different camera pairs by stereo matching and then fuses the depth maps to achieve robustness to mud spots, water droplets on camera lenses, and glare caused by bright light. Jin et al.…”

Section: Related Workmentioning

confidence: 99%

Effective fusion module with dilation convolution for monocular panoramic depth estimate

Han,

Cai,

Pan

et al. 2023

IET Image Processing

View full text Add to dashboard Cite

Depth estimation from monocular panoramic image is a crucial step in 3D reconstruction, which is a close relationship with virtual reality and metaverse technologies. In recent years, some methods, such as HRDFuse, BiFuse++, and UniFuse, have employed a two‐branch neural network leveraging two common projections: equirectangular and cubemap projections (CMPs). The equirectangular projection (ERP) provides a complete field of view but introduces distortion, while the CMP avoids distortion but introduces discontinuity at the boundary of the cube. In order to address the issue of distortion and discontinuity, the authors propose an efficient depth estimation fusion module to balance the feature mapping of the two projections. Moreover, for the ERP, the authors propose a novel inflated network architecture to extend the receptive field and effectively harness visual information. Extensive experiments show that the authors’ method predicts more clear boundaries and accurate depth results while outperforming mainstream panoramic depth estimation algorithms.

show abstract

FoVA-Depth: Field-of-View Agnostic Depth Estimation for Cross-Dataset Generalization

Lichy,

Su,

Badki

et al. 2024

2024 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

MODE: Multi-view Omnidirectional Depth Estimation with 360$$^\circ $$ Cameras

Cited by 5 publications

References 34 publications

CasOmniMVS: Cascade Omnidirectional Depth Estimation with Dynamic Spherical Sweeping

CasOmniMVS: Cascade Omnidirectional Depth Estimation with Dynamic Spherical Sweeping

Effective fusion module with dilation convolution for monocular panoramic depth estimate

FoVA-Depth: Field-of-View Agnostic Depth Estimation for Cross-Dataset Generalization

Contact Info

Product

Resources

About