360° Depth Estimation from Multiple Fisheye Images with Origami Crown Representation of Icosahedron

Komatsu, Ren; Fujii, Hiromitsu; Tamura, Yusuke; Yamashita, Atsushi; Asama, Hajime

doi:10.1109/iros45743.2020.9340981

Cited by 18 publications

(8 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In many works, the environment is directly captured by omnidirectional cameras [1], [20], [2], [3], and the neural network is used to implicitly learn the structural information of panoramic images to predict depth information. Some panorama work also uses multiple cameras [25], [12], [24], [26], [17], [14] to realize the depth perception of the panorama as shown in Fig. 2.…”

Section: A Multi-fisheye Omnidirectional Depth Estimationmentioning

confidence: 99%

Unsupervised OmniMVS: Efficient Omnidirectional Depth Inference via Establishing Pseudo-Stereo Supervision

Chen¹,

Lin²,

Nie³

et al. 2023

Preprint

View full text Add to dashboard Cite

Omnidirectional multi-view stereo (MVS) vision is attractive for its ultra-wide field-of-view (FoV), enabling machines to perceive 360°3D surroundings. However, the existing solutions require expensive dense depth labels for supervision, making them impractical in real-world applications.In this paper, we propose the first unsupervised omnidirectional MVS framework based on multiple fisheye images. To this end, we project all images to a virtual view center and composite two panoramic images with spherical geometry from two pairs of back-to-back fisheye images. The two 360°images formulate a stereo pair with a special pose, and the photometric consistency is leveraged to establish the unsupervised constraint, which we term "Pseudo-Stereo Supervision". In addition, we propose Un-OmniMVS, an efficient unsupervised omnidirectional MVS network, to facilitate the inference speed with two efficient components. First, a novel feature extractor with frequency attention is proposed to simultaneously capture the non-local Fourier features and local spatial features, explicitly facilitating the feature representation. Then, a variance-based light cost volume is put forward to reduce the computational complexity. Experiments exhibit that the performance of our unsupervised solution is competitive to that of the state-ofthe-art (SoTA) supervised methods with better generalization in real-world data. The code will be available at https: //github.com/Chen-z-s/Un-OmniMVS.

show abstract

Section: A Multi-fisheye Omnidirectional Depth Estimationmentioning

confidence: 99%

Unsupervised OmniMVS: Efficient Omnidirectional Depth Inference via Establishing Pseudo-Stereo Supervision

Chen¹,

Lin²,

Nie³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Komatsu et al [11] present IcoSweepNet for depth estimation from four omnidirectional images. IcoSweepNet based on an icosahedron representation, spherical sweeping and a 2D/3D CNN architecture called CrownConv, which is specially designed for extracting features of icosahedrons.…”

Section: Related Workmentioning

confidence: 99%

OmniGlasses: An Optical Aid for Stereo Vision CNNs to Enable Omnidirectional Image Processing

Seuffert

Grassi

Ahmed

et al. 2023

Preprint

View full text Add to dashboard Cite

Stereo vision is a key technology for 3D scene reconstruction from image pairs. Most approaches process perspective images from commodity cameras. These images, however, have a very limited field of view and only picture a small portion of the scene. In contrast, omnidirectional images, also known as fisheye images, exhibit a much larger field of view and allow a full 3D scene reconstruction with a small amount of cameras if placed carefully. However, omnidirectional images are strongly distorted which make the 3D reconstruction much more sophisticated. Nowadays, a lot of research is conducted on CNNs for omnidirectional stereo vision. Nevertheless, a significant gap between estimation accuracy and throughput can be observed in the literature. This work aims to bridge this gap by introducing a novel set of transformations, namely OmniGlasses. These are incorporated into the architecture of a fast network, i.e., AnyNet, originally designed for scene reconstruction on perspective images. Our network, Omni-AnyNet, produces accurate omnidirectional distance maps with a mean absolute error of around 13 cm at 36.4 fps and is therefore real-time capable.

show abstract

“…Recently, a lot of researches on using a convolutional neural network to cope with a GDD image has been reported [12][13][14][15]. In Ref.…”

Section: Geodesic-dome-division-based Convolutionmentioning

confidence: 99%

“…Recently, in Ref. [15], a method for all-around depth estimation from multiple omnidirectional images for indoor environment is proposed, in which an origami crown representation of icosahedron is implemented. However, carrying out convolution directly on a GDD image often costs a much longer processing time.…”

Section: Geodesic-dome-division-based Convolutionmentioning

confidence: 99%

Discrete Spherical Image Representation for CNN‐Based Spherical Camera Pose Estimation

Wang

2021

IEEJ Transactions Elec Engng

View full text Add to dashboard Cite

A spherical image has a full field of view. Using spherical images can estimate camera pose more accurately than conventional perspective images. It is known that a sphere can be quasi-uniform sampled by dividing a geodesic dome repeatedly; it results in a representation with rotational invariance approximately. Based on this fact, in this paper we proposed a method of estimating a spherical camera pose based on the geodesic-dome-division-based discrete spherical image representation by modifying the conventional perspective-image-oriented convolutional neural network, called PoseNet. First, a spherical image is represented as a geodesic-dome-division-based discrete spherical image. Then, to cope with the six-connectivity of the pixels of a geodesic-domedivision-based discrete spherical image using the conventional convolutional neural network framework, the discrete spherical image is flattened as a connected five parallelograms and resampled as an eight-connectivity image. Consequently, via the comparative experiments, it is found that the geodesic-dome-division-based discrete spherical image representation approach can achieve a better result of camera pose estimation than equirectangular image and CubeMap representations.

show abstract

360° Depth Estimation from Multiple Fisheye Images with Origami Crown Representation of Icosahedron

Cited by 18 publications

References 24 publications

Unsupervised OmniMVS: Efficient Omnidirectional Depth Inference via Establishing Pseudo-Stereo Supervision

Unsupervised OmniMVS: Efficient Omnidirectional Depth Inference via Establishing Pseudo-Stereo Supervision

OmniGlasses: An Optical Aid for Stereo Vision CNNs to Enable Omnidirectional Image Processing

Discrete Spherical Image Representation for CNN‐Based Spherical Camera Pose Estimation

Contact Info

Product

Resources

About