GANVO: Unsupervised Deep Monocular Visual Odometry and Depth Estimation with Generative Adversarial Networks

Almalioglu, Yasin; Saputra, Muhamad Risqi U.; Gusmão, Pedro P. B. de; Markham, Andrew; Trigoni, Niki

doi:10.1109/icra.2019.8793512

Cited by 141 publications

(93 citation statements)

References 39 publications

(60 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Years Training set Sup Semi-sup Unsup Main contributions GAN, LSTM, mask Almalioglu et al [91] Mono. sequences √ GAN, LSTM Table 2 Monocular depth results of semi-supervised and unsupervised methods on the KITTI dataset [32].…”

Section: Supervised (Sup) Manner Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Monocular depth estimation based on deep learning: An overview

Zhao¹,

Sun²,

Zhang³

et al. 2020

Sci. China Technol. Sci.

219

View full text Add to dashboard Cite

Depth information is important for autonomous systems to perceive environments and estimate their own state. Traditional depth estimation methods, like structure from motion and stereo vision matching, are built on feature correspondences of multiple viewpoints. Meanwhile, the predicted depth maps are sparse. Inferring depth information from a single image (monocular depth estimation) is an ill-posed problem. With the rapid development of deep neural networks, monocular depth estimation based on deep learning has been widely studied recently and achieved promising performance in accuracy. Meanwhile, dense depth maps are estimated from single images by deep neural networks in an end-to-end manner. In order to improve the accuracy of depth estimation, different kinds of network frameworks, loss functions and training strategies are proposed subsequently. Therefore, we survey the current monocular depth estimation methods based on deep learning in this review. Initially, we conclude several widely used datasets and evaluation indicators in deep learning-based depth estimation. Furthermore, we review some representative existing methods according to different training manners: supervised, unsupervised and semi-supervised. Finally, we discuss the challenges and provide some ideas for future researches in monocular depth estimation.

show abstract

Section: Supervised (Sup) Manner Methodsmentioning

confidence: 99%

“…In refs. [83,90,91], the generator consists of a pose network and a depth network, and the output of networks is used to synthesize images by view reconstruction. Then, a discriminator is designed to distinguish the real and predicted depth maps.…”

Section: Unsupervised Monocular Depth Estimationmentioning

confidence: 99%

Monocular depth estimation based on deep learning: An overview

Zhao¹,

Sun²,

Zhang³

et al. 2020

Sci. China Technol. Sci.

219

View full text Add to dashboard Cite

show abstract

“…Recently, another interesting approach to estimate the 6-DoF pose was published by Almalioglu et al [1]. They predict the pose of consecutive frames from different perspectives with a Generative Adversarial Network (GAN).…”

Section: Deep Learning Methodsmentioning

confidence: 99%

DistanceNet: Estimating Traveled Distance From Monocular Images Using a Recurrent Convolutional Neural Network

Kreuzig

Ochs

Mester

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

Classical monocular vSLAM/VO methods suffer from the scale ambiguity problem. Hybrid approaches solve this problem by adding deep learning methods, for example by using depth maps which are predicted by a CNN. We suggest that it is better to base scale estimation on estimating the traveled distance for a set of subsequent images. In this paper, we propose a novel end-to-end many-to-one traveled distance estimator. By using a deep recurrent convolutional neural network (RCNN), the traveled distance between the first and last image of a set of consecutive frames is estimated by our DistanceNet. Geometric features are learned in the CNN part of our model, which are subsequently used by the RNN to learn dynamics and temporal information. Moreover, we exploit the natural order of distances by using ordinal regression to predict the distance. The evaluation on the KITTI dataset shows that our approach outperforms current state-of-the-art deep learning pose estimators and classical mono vSLAM/VO methods in terms of distance prediction. Thus, our DistanceNet can be used as a component to solve the scale problem and help improve current and future classical mono vSLAM/VO methods.

show abstract

“…To reduce the dependency of labeled data, Zhou et al [10] propose an unsupervised method to predict image depth together with ego-motion using two networks, then compute the re-projected image residual as the loss function. After that, many enhanced works are proposed by adding additional 3D geometric loss [11], binocular loss [12] , deep feature reconstruction loss [13], dynamic and optical flow loss [14] or adversarial loss [15]. To achieve robustness of the system, Klodt et al [16] and Yang et al [17] further propose to estimate uncertainty of estimated ego-motion and depth.…”

Section: Introductionmentioning

confidence: 99%

Deep Monocular Visual Odometry for Ground Vehicle

Wang

Zhang

2020

IEEE Access

View full text Add to dashboard Cite

Monocular visual odometry, with the ability to help robots to locate themselves in unexplored environments, has been a crucial research problem in robotics. Though the existed learning-based endto-end methods can reduce engineering efforts such as accurate camera calibration and tedious case-bycase parameter tuning, the accuracy is still limited. One of the main reasons is that previous works aim to learn six-degrees-of-freedom motions despite the constrained motion of a ground vehicle by its mechanical structure and dynamics. To push the limit, we analyze the motion pattern of a ground vehicle and focus on learning two-degrees-of-freedom motions by proposed motion focusing and decoupling. The experiments on KITTI dataset show that the proposed motion focusing and decoupling approach can improve the visual odometry performance by reducing the relative pose error. Moreover, with the dimension reduction of the learning objective, our network is much lighter with only four convolution layers, which can quickly converge during the training stage and run in real-time at over 200 frames per second during the testing stage.

show abstract

GANVO: Unsupervised Deep Monocular Visual Odometry and Depth Estimation with Generative Adversarial Networks

Cited by 141 publications

References 39 publications

Monocular depth estimation based on deep learning: An overview

Monocular depth estimation based on deep learning: An overview

DistanceNet: Estimating Traveled Distance From Monocular Images Using a Recurrent Convolutional Neural Network

Deep Monocular Visual Odometry for Ground Vehicle

Contact Info

Product

Resources

About