Beyond Photometric Loss for Self-Supervised Ego-Motion Estimation

Shen, Tianwei; Luo, Zixin; Zhou, Lei; Deng, Han‐Yu; Zhang, Runze; Fang, Tian; Quan, Long

doi:10.1109/icra.2019.8793479

Cited by 69 publications

(47 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For different scenes with different sensors, these methods are difficult to transfer because sensors should be photometrically recalibrated, and also correct uncertainty map formation for matching points is required. Modern enhancements of these approaches are neural network methods that train in a self-supervised manner -D3VO [35], Deep-MatchVO [36], DF-VO [37]. All of them allow generating pose estimation of two neighbor frames a monocular camera and depth map.…”

Section: Visual-based Robot Localizationmentioning

confidence: 99%

Real-Time Object Navigation With Deep Neural Networks and Hierarchical Reinforcement Learning

et al. 2020

View full text Add to dashboard Cite

In the last years, deep learning and reinforcement learning methods have significantly improved mobile robots in such fields as perception, navigation, and planning. But there are still gaps in applying these methods to real robots due to the low computational efficiency of recent neural network architectures and their poor adaptability to robotic experiments' realities. In this paper, we consider an important task in mobile robotics -navigation to an object using an RGB-D camera. We develop a new neural network framework for robot control that is fast and resistant to possible noise in sensors and actuators. We propose an original integration of semantic segmentation, mapping, localization, and reinforcement learning methods to improve the effectiveness of exploring the environment, finding the desired object, and quickly navigating to it. We created a new HISNav dataset based on the Habitat virtual environment, which allowed us to use simulation experiments to pre-train the model and then upload it to a real robot. Our architecture is adapted to work in a real-time environment and fully implements modern trends in this area.

show abstract

Section: Visual-based Robot Localizationmentioning

confidence: 99%

Real-Time Object Navigation With Deep Neural Networks and Hierarchical Reinforcement Learning

et al. 2020

View full text Add to dashboard Cite

show abstract

“…where is the threshold factor that controls the selecting capability of the mask, we set it to 0.45 in our experimental settings. We also strengthen both masks by filtering out the cross-frame static pixels and non-principal area [22,40], which are commonly used in several state-of-the-art methods [22,26,40,52].…”

Section: Combined Selective Maskmentioning

confidence: 99%

“…The training of depth and ego-motion networks are entangled during self-supervision [52,74]. Thus, enhancing the ego-motion estimation by extra supervision is also beneficial for depth estimation.…”

Section: Cycle Consistency Constraintmentioning

confidence: 99%

Enhancing Self-supervised Monocular Depth Estimation via Incorporating Robust Constraints

Zhu

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Self-supervised depth estimation has shown great prospects in inferring 3D structures using purely unannotated images. However, its performance usually drops when trained on the images with changing brightness and moving objects. In this paper, we address this issue by enhancing the robustness of the self-supervised paradigm using a set of image-based and geometry-based constraints. Our contributions are threefold, 1) we propose a gradient-based robust photometric loss which restrains the false supervisory signals caused by brightness changes, 2) we propose to filter out the unreliable areas that violate the rigid assumption by a novel combined selective mask, which is computed on the forward pass of the network by leveraging the inter-loss consistency and the lossgradient consistency, and 3) we constrain the motion estimation network to generate across-frame consistent motions via proposing a triplet-based cycle consistency constraint. Extensive experiments conducted on KITTI, Cityscape and Make3D datasets demonstrate the superiority of our method, that the proposed method can effectively handle complex scenes with changing brightness and object motions. Both qualitative and quantitative results show that the proposed method outperforms the state-of-the-art methods. CCS CONCEPTS • Computing methodologies → 3D imaging; Motion capture.

show abstract

“…Previous works employ an additional branch to regress an uncertainty map, which helps a little [54]. Instead, we follow the explicit occlusion modeling approach [41] which does not rely on the data-driven uncertainty. We have observed that photometric inconsistency such as moving objects usually incurs larger photometric errors (Figure 4(b)).…”

Section: Differentiable Sparse Feature Selectionmentioning

confidence: 99%

Self-Supervised Learning of Depth and Motion Under Photometric Inconsistency

Shen

Zhou

Luo

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

Self Cite

View full text Add to dashboard Cite

The self-supervised learning of depth and pose from monocular sequences provides an attractive solution by using the photometric consistency of nearby frames as it depends much less on the ground-truth data. In this paper, we address the issue when previous assumptions of the self-supervised approaches are violated due to the dynamic nature of real-world scenes. Different from handling the noise as uncertainty, our key idea is to incorporate more robust geometric quantities and enforce internal consistency in the temporal image sequence. As demonstrated on commonly used benchmark datasets, the proposed method substantially improves the state-of-the-art methods on both depth and relative pose estimation for monocular image sequences, without adding inference overhead.

show abstract

Beyond Photometric Loss for Self-Supervised Ego-Motion Estimation

Cited by 69 publications

References 48 publications

Real-Time Object Navigation With Deep Neural Networks and Hierarchical Reinforcement Learning

Real-Time Object Navigation With Deep Neural Networks and Hierarchical Reinforcement Learning

Enhancing Self-supervised Monocular Depth Estimation via Incorporating Robust Constraints

Self-Supervised Learning of Depth and Motion Under Photometric Inconsistency

Contact Info

Product

Resources

About