Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments

Barnes, Dan; Maddern, Will; Pascoe, Geoffrey; Posner, Ingmar

doi:10.1109/icra.2018.8460564

Cited by 63 publications

(54 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, these approaches commonly handle non-rigid objects within a relative static background, which is out of major scope of this paper. Most recently, Barnes et al [78] show that explicitly modeling moving things with a 3D prior map can avoid visual odometry drifting. We also consider moving object segmentation, which is under an unsupervised setting with videos only.…”

Section: Related Workmentioning

confidence: 99%

Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding

Luo

Yang

Wang³

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

237

161

View full text Add to dashboard Cite

Learning to estimate 3D geometry in a single frame and optical flow from consecutive frames by watching unlabeled videos via deep convolutional network has made significant progress recently. Current state-of-the-art (SoTA) methods treat the two tasks independently. One typical assumption of the existing depth estimation methods is that the scenes contain no independent moving objects. while object moving could be easily modeled using optical flow. In this paper, we propose to address the two tasks as a whole, i.e. to jointly understand per-pixel 3D geometry and motion. This eliminates the need of static scene assumption and enforces the inherent geometrical consistency during the learning process, yielding significantly improved results for both tasks. We call our method as "Every Pixel Counts++" or "EPC++". Specifically, during training, given two consecutive frames from a video, we adopt three parallel networks to predict the camera motion (MotionNet), dense depth map (DepthNet), and per-pixel optical flow between two frames (OptFlowNet) respectively. The three types of information, are fed into a holistic 3D motion parser (HMP), and per-pixel 3D motion of both rigid background and moving objects are disentangled and recovered. Various loss terms are formulated to jointly supervise the three networks. An effective adaptive training strategy is proposed to achieve better performance and more efficient convergence. Comprehensive experiments were conducted on datasets with different scenes, including driving scenario (KITTI 2012 and KITTI 2015 datasets), mixed outdoor/indoor scenes (Make3D) and synthetic animation (MPI Sintel dataset). Performance on the five tasks of depth estimation, optical flow estimation, odometry, moving object segmentation and scene flow estimation shows that our approach outperforms other SoTA methods, demonstrating the effectiveness of each module of our proposed method. Code will be available at: https://github.com/chenxuluo/EPC.

show abstract

Section: Related Workmentioning

confidence: 99%

Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding

Luo

Yang

Wang³

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

237

161

View full text Add to dashboard Cite

show abstract

“…Whereas single-view depth estimation and multi-view geometry are mostly taken as individual tasks, a few works combine both. The single-view depth estimation can be useful for scale estimation in monocular visual odometry [2,69,71] or fused with SfM-based depth estimates in static environments [12,54,71]. Kumar et al [37] used singleview depth estimation for depth initialization in a multibody or non-rigid SfM-based approach similar to [36].…”

Section: Related Workmentioning

confidence: 99%

“…The key idea applied here is to integrate single-view depth information to provide the metric scale. In contrast to [2,69,71], we apply this idea additionally for scale-aware pose estimation of moving objects. First, object instances in the images I 0 and I 1 detected by a Mask R-CNN [23] (implementation of [63]) are paired based on sparse flow correspondences (p i 0 , p i 1 ) [18] using a simple voting scheme.…”

Section: Monocular Scene Flowmentioning

confidence: 99%

Mono-SF: Multi-View Geometry Meets Single-View Depth for Monocular Scene Flow Estimation of Dynamic Traffic Scenes

Brickwedde

Abraham

Mester

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Existing 3D scene flow estimation methods provide the 3D geometry and 3D motion of a scene and gain a lot of interest, for example in the context of autonomous driving. These methods are traditionally based on a temporal series of stereo images. In this paper, we propose a novel monocular 3D scene flow estimation method, called Mono-SF. Mono-SF jointly estimates the 3D structure and motion of the scene by combining multi-view geometry and singleview depth information. Mono-SF considers that the scene flow should be consistent in terms of warping the reference image in the consecutive image based on the principles of multi-view geometry. For integrating single-view depth in a statistical manner, a convolutional neural network, called ProbDepthNet, is proposed. ProbDepthNet estimates pixelwise depth distributions from a single image rather than single depth values. Additionally, as part of ProbDepth-Net, a novel recalibration technique for regression problems is proposed to ensure well-calibrated distributions. Our experiments show that Mono-SF outperforms state-of-theart monocular baselines and ablation studies support the Mono-SF approach and ProbDepthNet design.

show abstract

“…Deep learning can be used to improve the accuracy of VO by directly influencing the precision of the keypoints detector. In Barnes, Maddern, Pascoe, and Posner (), a deep neural network has been trained for learning keypoints distractors in monocular VO. The so‐called learned ephemerality mask acts as a rejection scheme for keypoints outliers which might decrease the vehicle localization's accuracy.…”

Section: Deep Learning For Driving Scene Perception and Localizationmentioning

confidence: 99%

A survey of deep learning techniques for autonomous driving

Grigorescu

Trasnea

Cocias

et al. 2019

Journal of Field Robotics

1,127

495

View full text Add to dashboard Cite

The last decade witnessed increasingly rapid progress in self‐driving vehicle technology, mainly backed up by advances in the area of deep learning and artificial intelligence (AI). The objective of this paper is to survey the current state‐of‐the‐art on deep learning technologies used in autonomous driving. We start by presenting AI‐based self‐driving architectures, convolutional and recurrent neural networks, as well as the deep reinforcement learning paradigm. These methodologies form a base for the surveyed driving scene perception, path planning, behavior arbitration, and motion control algorithms. We investigate both the modular perception‐planning‐action pipeline, where each module is built using deep learning methods, as well as End2End systems, which directly map sensory information to steering commands. Additionally, we tackle current challenges encountered in designing AI architectures for autonomous driving, such as their safety, training data sources, and computational hardware. The comparison presented in this survey helps gain insight into the strengths and limitations of deep learning and AI approaches for autonomous driving and assist with design choices.

show abstract

Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments

Cited by 63 publications

References 34 publications

Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding

Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding

Mono-SF: Multi-View Geometry Meets Single-View Depth for Monocular Scene Flow Estimation of Dynamic Traffic Scenes

A survey of deep learning techniques for autonomous driving

Contact Info

Product

Resources

About