Dense 3D reconstruction from two spherical images via optical flow-based equirectangular epipolar rectification

Pathak, Sarthak; Moro, Alessandro; Yamashita, Atsushi; Asama, Hajime

doi:10.1109/ist.2016.7738212

Cited by 17 publications

(19 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The 3D reconstruction method described in this paper is based on the approach explained in [1]. The technique relies on the fact that spherical images contain data from all directions, which can be exploited to estimate the camera motion between two such images.…”

Section: D Reconstruction Using the Optical Flow Between Spherical Imagesmentioning

confidence: 99%

“…The goal of our research is to develop software that provides a semantically meaningful map from a pair of spherical images and works as close to real time as possible, and thus the accuracy of the 3D coordinates is given less emphasis than the processing time. Our work is mainly connected to the dense reconstruction method presented in [1], which describes how to use the optical flow between two spherical images to estimate the motion between the two camera poses and how to utilize this information to build a 3D model with an arbitrary metric scale. The follow-up work in [2] discusses the motion estimation when dealing with more than two images.…”

Section: Introductionmentioning

confidence: 99%

“…Compared to all of these methods, the most distinct feature of our approach is using semantic information to increase the robustness of the motion evaluation, particularly in outdoor environments. The main difference compared to [1] is that the camera motion is evaluated solely based on the regions where we except to obtain a reliable estimation of the optical flow. The decision whether to include a region in the motion evaluation is based on its semantic classification.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

All-Around 3D Reconstruction from Spherical Images with Semantic Segmentation

Pennanen

Ariram

Tikanmäki

et al. 2021

2021 IEEE International Conference on Mechatronics and Automation (ICMA)

View full text Add to dashboard Cite

Lightweight, affordable spherical cameras can be utilized in mobile robotics to build a 3D map of the working environment of a robot. This paper demonstrates how to use the optical flow between two spherical images to quickly construct a semantically meaningful 3D model spanning all directions. The main contribution of the paper is the use of semantic segmentation to increase the robustness of the reconstruction method in both indoor and outdoor applications.

show abstract

Section: D Reconstruction Using the Optical Flow Between Spherical Imagesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

All-Around 3D Reconstruction from Spherical Images with Semantic Segmentation

Pennanen

Ariram

Tikanmäki

et al. 2021

2021 IEEE International Conference on Mechatronics and Automation (ICMA)

View full text Add to dashboard Cite

show abstract

“…Reference [8] rotated spherical images to avoid the distorted areas, and extracted robust features from the central region of the image. Reference [9] rectified the optical flow in an equirectangular projection by rotating the images on a sphere to estimate the motion of the spherical camera. These methods are based on the fact that spherical images contain all-round information and can be rotated without any loss of information.…”

Section: Introductionmentioning

confidence: 99%

SelfSphNet: Motion Estimation of a Spherical Camera via Self-Supervised Learning

et al. 2020

Self Cite

View full text Add to dashboard Cite

In this paper, we propose SelfSphNet, that is, a self-supervised learning network to estimate the motion of an arbitrarily moving spherical camera without the need for any labeled training data. Recently, numerous learning-based methods for camera motion estimation have been proposed. However, most of these methods require an enormous amount of labeled training data, which is difficult to acquire experimentally. To solve this problem, our SelfSphNet employs two loss functions to estimate the frame-to-frame camera motion, thus giving two supervision signals to the network with the usage of unlabeled training data. First, a 5 DoF epipolar angular loss, which is composed of a dense optical flow of spherical images, estimates the 5 DoF motion between two image frames. This loss function utilizes a unique property of the spherical optical flow, which allows the rotational and translational components to be decoupled by using a derotation operation. This operation is derived from the fact that spherical images can be rotated to any orientation without any loss of information, hence making it possible to ''decouple'' the dense optical flow between pairs of spherical images to a pure translational state. Next, a photometric reprojection loss estimates the full 6 DoF motion using a depth map generated from the decoupled optical flow. This minimization strategy enables our network to be optimized without using any labeled training data. To confirm the effectiveness of our proposed approach (SelfSphNet), several experiments to estimate the camera trajectory, as well as the camera motion, were conducted in comparison to a previous self-supervised learning approach, SfMLearner, and a fully supervised learning approach whose baseline network is the same as SelfSphNet. Moreover, transfer learning in a new scene was also conducted to verify that our proposed method can optimize the network with newly collected unlabeled data. INDEX TERMS Motion estimation, computer vision, image processing, deep learning, convolutional neural networks.

show abstract

“…Image-based 3D scene reconstruction approaches have been widely studied by the scientific community, having applications in archaeological [1] and architectural modeling [2], robot navigation [3], autonomous driving systems [4], and infrastructure inspection [5], just to mention a few. Most existing techniques deal with the traditional pinhole-based/perspective cameras, which present a narrow field of view (FoV), and, hence, require several captures to model large scenes.…”

Section: Introductionmentioning

confidence: 99%

Dense 3D Indoor Scene Reconstruction from Spherical Images

Silveira¹,

Jung

2020

Anais Estendidos Da Conference on Graphics, Patterns and Images (SIBRAPI Estendido 2020)

View full text Add to dashboard Cite

Techniques for 3D reconstruction of scenes based on images are popular and support a number of secondary applications. Traditional approaches require several captures for covering whole environments due to the narrow field of view (FoV) of the pinhole-based/perspective cameras. This paper summarizes the main contributions of the homonym Ph.D. Thesis, which addresses the 3D scene reconstruction problem by considering omnidirectional (spherical or 360◦ ) cameras that present a 360◦ × 180◦ FoV. Although spherical imagery have the benefit of the full-FoV, they are also challenging due to the inherent distortions involved in the capture and representation of such images, which might compromise the use of many wellestablished algorithms for image processing and computer vision. The referred Ph.D. Thesis introduces novel methodologies for estimating dense depth maps from two or more uncalibrated and temporally unordered 360◦ images. It also presents a framework for inferring depth from a single spherical image. We validate our approaches using both synthetic data and computer-generated imagery, showing competitive results concerning other state-ofthe-art methods.

show abstract

Dense 3D reconstruction from two spherical images via optical flow-based equirectangular epipolar rectification

Cited by 17 publications

References 15 publications

All-Around 3D Reconstruction from Spherical Images with Semantic Segmentation

All-Around 3D Reconstruction from Spherical Images with Semantic Segmentation

SelfSphNet: Motion Estimation of a Spherical Camera via Self-Supervised Learning

Dense 3D Indoor Scene Reconstruction from Spherical Images

Contact Info

Product

Resources

About