3D reconstruction of structures using spherical cameras with small motion

Pathak, Sarthak; Moro, Alessandro; Fujii, Hiromitsu; Yamashita, Atsushi; Asama, Hajime

doi:10.1109/iccas.2016.7832307

Cited by 15 publications

(10 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are many optical low algorithms designed to work with perspective imagery (refer to [180] for a recent survey). Similarly to sparse feature matching, some works [39,120,172] adopt traditional optical low approaches for 360 • image processing, sometimes in between pre-and post-processing steps. The most straightforward workaround found in literature is to circularly pad ERP images before computing the optical low, avoiding longitudinal disconnection [39,172] (recall that a similar strategy was adopted in the context of deep learning for depth or layout estimation, as mentioned in Section 3).…”

Section: Dense Feature Extraction and Matchingmentioning

confidence: 99%

See 1 more Smart Citation

3D Scene Geometry Estimation from 360° Imagery: A Survey

Silveira¹,

Pinto²,

Murrugarra-Llerena³

et al. 2022

ACM Comput. Surv.

View full text Add to dashboard Cite

This paper provides a comprehensive survey on pioneer and state-of-the-art 3D scene geometry estimation methodologies based on single, two, or multiple images captured under omnidirectional optics. We first revisit the basic concepts of the spherical camera model and review the most common acquisition technologies and representation formats suitable for omnidirectional (also called 360°, spherical or panoramic) images and videos. We then survey monocular layout and depth inference approaches, highlighting the recent advances in learning-based solutions suited for spherical data. The classical stereo matching is then revised on the spherical domain, where methodologies for detecting and describing sparse and dense features become crucial. The stereo matching concepts are then extrapolated for multiple view camera setups, categorizing them among light fields, multi-view stereo, and structure from motion (or visual simultaneous localization and mapping). We also compile and discuss commonly adopted datasets and figures of merit indicated for each purpose and list recent results for completeness. We conclude this paper by pointing out current and future trends.

show abstract

Section: Dense Feature Extraction and Matchingmentioning

confidence: 99%

“…The authors also present hints on how additional cameras can further improve the results. Pathak et al [120,123] also allow small baseline and arbitrary rotation. They irst estimate the 5-DoF pose between the images using A-KAZE features and RANSAC-enabled 8-PA [71], and then łderotatež the images.…”

Section: Depth Estimationmentioning

confidence: 99%

3D Scene Geometry Estimation from 360° Imagery: A Survey

Silveira¹,

Pinto²,

Murrugarra-Llerena³

et al. 2022

ACM Comput. Surv.

View full text Add to dashboard Cite

show abstract

“…With the entire field of view, 360 • camera is more robust to rotation and translation movements compared to normal perspective camera [22]. There are some recent methods aim to tackle 360 • perception tasks.…”

Section: Related Workmentioning

confidence: 99%

Self-supervised Learning of Depth and Camera Motion from 360 $$^\circ $$ Videos

Wang

Cheng

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

As 360 • cameras become prevalent in many autonomous systems (e.g., self-driving cars and drones), efficient 360 • perception becomes more and more important. We propose a novel self-supervised learning approach for predicting the omnidirectional depth and camera motion from a 360 • video. In particular, starting from the SfMLearner, which is designed for cameras with normal field-of-view, we introduce three key features to process 360 • images efficiently. Firstly, we convert each image from equirectangular projection to cubic projection in order to avoid image distortion. In each network layer, we use Cube Padding (CP), which pads intermediate features from adjacent faces, to avoid image boundaries. Secondly, we propose a novel "spherical" photometric consistency constraint on the whole viewing sphere. In this way, no pixel will be projected outside the image boundary which typically happens in images with normal field-of-view. Finally, rather than naively estimating six independent camera motions (i.e., naively applying SfM-Learner to each face on a cube), we propose a novel camera pose consistency loss to ensure the estimated camera motions reaching consensus. To train and evaluate our approach, we collect a new PanoSUNCG dataset containing a large amount of 360 • videos with groundtruth depth and camera motion. Our approach achieves state-of-the-art depth prediction and camera motion estimation on PanoSUNCG with faster inference speed comparing to equirectangular. In real-world indoor videos, our approach can also achieve qualitatively reasonable depth prediction by acquiring model pre-trained on PanoSUNCG.

show abstract

“…machine vision applications, including scene reconstruction [8,23,27,30,31,33,34,36,44], view synthesis [20,26], navigation [42,43,46], and tracking [3,21].…”

Section: Introductionmentioning

confidence: 99%

Dense Depth Estimation from Multiple 360-degree Images Using Virtual Depth

Yang,

Kim,

Lee

2021

Preprint

View full text Add to dashboard Cite

In this paper, we propose a dense depth estimation pipeline for multiview 360°images. The proposed pipeline leverages a spherical camera model that compensates for radial distortion in 360°images. The key contribution of this paper is the extension of a spherical camera model to multiview by introducing a translation scaling scheme. Moreover, we propose an effective dense depth estimation method by setting virtual depth and minimizing photonic reprojection error. We validate the performance of the proposed pipeline using the images of natural scenes as well as the synthesized dataset for quantitive evaluation. The experimental results verify that the proposed pipeline improves estimation accuracy compared to the current state-of-art dense depth estimation methods.

show abstract

3D reconstruction of structures using spherical cameras with small motion

Cited by 15 publications

References 15 publications

3D Scene Geometry Estimation from 360° Imagery: A Survey

3D Scene Geometry Estimation from 360° Imagery: A Survey

Self-supervised Learning of Depth and Camera Motion from 360 $$^\circ $$ Videos

Dense Depth Estimation from Multiple 360-degree Images Using Virtual Depth

Contact Info

Product

Resources

About