VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry

Radwan, Noha; Valada, Abhinav; Burgard, Wolfram

doi:10.1109/lra.2018.2869640

Cited by 233 publications

(156 citation statements)

References 3 publications

Supporting

Mentioning

149

Contrasting

Order By: Relevance

“…Neural networks, such as PoseNet (Kendall, Grimes, & Cipolla, 2015), VLocNet++ (Radwan, Valada, & Burgard, 2018) Although much progress has been reported in the area of deep learning-based localization, VO techniques are still dominated by classical keypoints matching algorithms, combined with acceleration data provided by inertial sensors. This is mainly due to the fact that keypoints detectors are computational efficient and can be easily deployed on embedded devices.…”

Section: Localizationmentioning

confidence: 99%

“…, or the approaches introduced inWalch et al (2017),Melekhov, Ylioinas, Kannala, and Rahtu (2017),Laskar, Melekhov, Kalia, and Kannala (2017),Brachmann and Rother (2018), or Sarlin, Debraine, Dymczyk, Siegwart, andCadena (2018), are using image data to estimate the 3D pose of a camera in an End2End fashion. Scene semantics can be derived together with the estimated pose(Radwan et al, 2018).LiDAR intensity maps are also suited for learning a real-time, calibration-agnostic localization for autonomous cars (Barsan, Wang, Pokrovsky, & Urtasun, 2018). The method uses a deep neural network to build a learned representation of the driving scene from LiDAR sweeps and intensity maps.…”

mentioning

confidence: 99%

See 1 more Smart Citation

A survey of deep learning techniques for autonomous driving

Grigorescu

Trasnea

Cocias

et al. 2019

Journal of Field Robotics

1,106

495

View full text Add to dashboard Cite

The last decade witnessed increasingly rapid progress in self‐driving vehicle technology, mainly backed up by advances in the area of deep learning and artificial intelligence (AI). The objective of this paper is to survey the current state‐of‐the‐art on deep learning technologies used in autonomous driving. We start by presenting AI‐based self‐driving architectures, convolutional and recurrent neural networks, as well as the deep reinforcement learning paradigm. These methodologies form a base for the surveyed driving scene perception, path planning, behavior arbitration, and motion control algorithms. We investigate both the modular perception‐planning‐action pipeline, where each module is built using deep learning methods, as well as End2End systems, which directly map sensory information to steering commands. Additionally, we tackle current challenges encountered in designing AI architectures for autonomous driving, such as their safety, training data sources, and computational hardware. The comparison presented in this survey helps gain insight into the strengths and limitations of deep learning and AI approaches for autonomous driving and assist with design choices.

show abstract

Section: Localizationmentioning

confidence: 99%

mentioning

confidence: 99%

A survey of deep learning techniques for autonomous driving

Grigorescu

Trasnea

Cocias

et al. 2019

Journal of Field Robotics

1,106

495

View full text Add to dashboard Cite

show abstract

“…Starting from this work, additional improvements have been proposed by introducing new geometric loss functions [15], by exploiting the uncertainty estimation of Bayesian CNNs [16], by including a data augmentation scheme based on synthetic depth information [17], or using the relative pose between two observations in a CNNs pipeline [18]. One of the many works that follow the idea presented in PoseNet is VLocNet++ [19].…”

Section: A Camera-only Approachesmentioning

confidence: 99%

CMRNet: Camera to LiDAR-Map Registration

Cattaneo¹,

Vaghi²,

Ballardini³

et al. 2019

2019 IEEE Intelligent Transportation Systems Conference (ITSC)

View full text Add to dashboard Cite

In this paper we present CMRNet, a realtime approach based on a Convolutional Neural Network (CNN) to localize an RGB image of a scene in a map built from LiDAR data. Our network is not trained in the working area, i.e., CMRNet does not learn the map. Instead it learns to match an image to the map. We validate our approach on the KITTI dataset, processing each frame independently without any tracking procedure. CMRNet achieves 0.27m and 1.07 • median localization accuracy on the sequence 00 of the odometry dataset, starting from a rough pose estimate displaced up to 3.5m and 17 • . To the best of our knowledge this is the first CNN-based approach that learns to match images from a monocular camera to a given, preexisting 3D LiDAR-map.

show abstract

“…However, unlike previous work focused on improving the retrieval [1,4,20,26,34,36] or matching [56,72], we focus on the pose verification stage, i.e., the problem of selecting the "best" pose from the n estimated poses. An alternative to the localization approaches outlined above is to train a CNN that directly regresses the camera pose from a given input image [9,12,32,33,50,78]. However, it was recently shown that such methods do not consistently outperform a simple image retrieval baseline [59].…”

Section: Related Workmentioning

confidence: 99%

Is This the Right Place? Geometric-Semantic Pose Verification for Indoor Visual Localization

Taira

Rocco

Sedlář

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Visual localization in large and complex indoor scenes, dominated by weakly textured rooms and repeating geometric patterns, is a challenging problem with high practical relevance for applications such as Augmented Reality and robotics. To handle the ambiguities arising in this scenario, a common strategy is, first, to generate multiple estimates for the camera pose from which a given query image was taken. The pose with the largest geometric consistency with the query image, e.g., in the form of an inlier count, is then selected in a second stage. While a significant amount of research has concentrated on the first stage, there is considerably less work on the second stage. In this paper, we thus focus on pose verification. We show that combining different modalities, namely appearance, geometry, and semantics, considerably boosts pose verification and consequently pose accuracy. We develop multiple hand-crafted as well as a trainable approach to join into the geometric-semantic verification and show significant improvements over stateof-the-art on a very challenging indoor dataset.

show abstract

VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry

Cited by 233 publications

References 3 publications

A survey of deep learning techniques for autonomous driving

A survey of deep learning techniques for autonomous driving

CMRNet: Camera to LiDAR-Map Registration

Is This the Right Place? Geometric-Semantic Pose Verification for Indoor Visual Localization

Contact Info

Product

Resources

About