Sequential Adversarial Learning for Self-Supervised Deep Visual Odometry

Li, Shunkai; Xue, Fei; Wang, Xin; Zhang, Yan; Zha, Hongbin

doi:10.1109/iccv.2019.00294

Cited by 54 publications

(44 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…[90,91]. Furthermore, Li et al [90] designed an additional network to eliminate the shortcoming of view reconstruction algorithm, which is similar to ref. [43].…”

Section: Unsupervised Monocular Depth Estimationmentioning

confidence: 98%

See 1 more Smart Citation

Monocular depth estimation based on deep learning: An overview

Zhao¹,

Sun²,

Zhang³

et al. 2020

Sci. China Technol. Sci.

223

View full text Add to dashboard Cite

Depth information is important for autonomous systems to perceive environments and estimate their own state. Traditional depth estimation methods, like structure from motion and stereo vision matching, are built on feature correspondences of multiple viewpoints. Meanwhile, the predicted depth maps are sparse. Inferring depth information from a single image (monocular depth estimation) is an ill-posed problem. With the rapid development of deep neural networks, monocular depth estimation based on deep learning has been widely studied recently and achieved promising performance in accuracy. Meanwhile, dense depth maps are estimated from single images by deep neural networks in an end-to-end manner. In order to improve the accuracy of depth estimation, different kinds of network frameworks, loss functions and training strategies are proposed subsequently. Therefore, we survey the current monocular depth estimation methods based on deep learning in this review. Initially, we conclude several widely used datasets and evaluation indicators in deep learning-based depth estimation. Furthermore, we review some representative existing methods according to different training manners: supervised, unsupervised and semi-supervised. Finally, we discuss the challenges and provide some ideas for future researches in monocular depth estimation.

show abstract

“…[90,91]. Furthermore, Li et al [90] designed an additional network to eliminate the shortcoming of view reconstruction algorithm, which is similar to ref. [43].…”

Section: Unsupervised Monocular Depth Estimationmentioning

confidence: 98%

“…In refs. [83,90,91], the generator consists of a pose network and a depth network, and the output of networks is used to synthesize images by view reconstruction. Then, a discriminator is designed to distinguish the real and predicted depth maps.…”

Section: Unsupervised Monocular Depth Estimationmentioning

confidence: 99%

Monocular depth estimation based on deep learning: An overview

Zhao¹,

Sun²,

Zhang³

et al. 2020

Sci. China Technol. Sci.

223

View full text Add to dashboard Cite

show abstract

“…We compare with recent self-supervised VO baselines: GeoNet [39], Vid2Depth [24], Zhan et al [41], SAVO [22] and Li et al [21] as well as classic methods: ORB-SLAM2 [26] (with and without loop closure) and VISO2 [14]. Besides, we compare with Zhao et al [43] and DF-VO [42] which are state-of-the-art methods that combine the output of pretrained networks with classic VO pipeline.…”

Section: Cityscapes To Kittimentioning

confidence: 99%

Self-Supervised Deep Visual Odometry With Online Adaptation

Wang

Cao

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

Despite learning-based visual odometry (VO) has shown impressive results in recent years, the pretrained networks may easily collapse in unseen environments. The large domain gap between training and testing data makes them difficult to generalize to new scenes. In this paper, we propose an online adaptation framework for deep VO with the assistance of scene-agnostic geometric computations and Bayesian inference. In contrast to learning-based pose estimation, our method solves pose from optical flow and depth while the single-view depth estimation is continuously improved with new observations by online learned uncertainties. Meanwhile, an online learned photometric uncertainty is used for further depth and pose optimization by a differentiable Gauss-Newton layer. Our method enables fast adaptation of deep VO networks to unseen environments in a self-supervised manner. Extensive experiments including Cityscapes to KITTI and outdoor KITTI to indoor TUM demonstrate that our method achieves state-of-the-art generalization ability among self-supervised VO methods.

show abstract

“…Similarly, most studies improved the optical flow performance by implementing deep learning rather than through fundamental improvements. In addition, there are studies that implement sequential adversarial learning for visual odometry using optical flow [11] and transfer learningbased visual tracking with gaussian processes regression using sequential image information similar to optical flow [12].…”

Section: Introductionmentioning

confidence: 99%

Improvement of Optical Flow Estimation by Using the Hampel Filter for Low-End Embedded Systems

Park

Lee

Suh

et al. 2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Owing to the recent advances in the field of deeplearning-based approaches, state-of-the-art performance has been achieved for optical flow estimation. However, non-deep-learningbased improvement in the optical flow estimation performance is still required because many platforms, such as small UGVs and drones, involve constraints that make mounting GPUs difficult. Thus, in this study, we do not apply deep learning methods; rather, we improve the accuracy of the optical flow pipeline by optimizing only its fundamental parameters. The optical flow estimation performance is influenced by the number of coarse-tofine estimation pyramids, the filter applied to each pyramid level, the window size, and the graduated nonconvexity (GNC) step number. No significant differences from the previous research were achieved by optimizing the parameters heuristically because they have already been optimized. Therefore, we decided to change the filter applied in each pyramid, which is the most important factor in determining the optical flow estimation performance. As a result of verification, the optical flow with the median filter did not show good optical flow estimation performance due to oversmoothing of the image boundary, and the optical flow with the weighted median filter proposed to overcome this drawback could not well address the complexity of the equation and deal with the large computation cost. The proposed Hampel filter shows better performance by minimizing the loss of the original image in the image smoothing process and reduced computational complexity compared to the weighted median filter. The principle of the Hampel filter is similar to that of the median filter, except that if the reference pixel is statistically close to the pixel median in the window, it works by preserving the original pixel value. This is usually used for filtering outliers in a 1D signal but has never been applied for 2D image filters. This study expanded the application of the existing Hampel filter to include 2D images. Compared to the conventional median filter, this filter demonstrates relatively good performance, as it can considerably reduce the loss in the image data and improve the excessive edge smoothing effect such that it can better identify motion. The performance-related improvements achieved by this research were verified using the KITTI Vision Benchmark Suite, and the simulation results demonstrate that the Hampel filter can replace the existing filter and that it has strengths related to multi-object detection, background recognition, and the identification of moving objects at the edges of images.

show abstract

Sequential Adversarial Learning for Self-Supervised Deep Visual Odometry

Cited by 54 publications

References 37 publications

Monocular depth estimation based on deep learning: An overview

Monocular depth estimation based on deep learning: An overview

Self-Supervised Deep Visual Odometry With Online Adaptation

Improvement of Optical Flow Estimation by Using the Hampel Filter for Low-End Embedded Systems

Contact Info

Product

Resources

About