Find your way by observing the sun and other semantic cues

Ma, Wei-Chiu; Wang, Shenlong; Brubaker, Marcus A.; Fidler, Sanja; Urtasun, Raquel

doi:10.1109/icra.2017.7989744

Cited by 41 publications

(59 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Residual networks [22] constitute the state-of-theart, as they allow to train very deep networks without problems of vanishing or exploding gradients. In the context of road classification, deep neural networks are also widely employed [37]. Sensor fusion has also been exploited in this context [50].…”

Section: Related Workmentioning

confidence: 99%

MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving

Teichmann¹,

Weber

Zöllner

et al. 2018

2018 IEEE Intelligent Vehicles Symposium (IV)

Self Cite

531

339

View full text Add to dashboard Cite

While most approaches to semantic reasoning have focused on improving performance, in this paper we argue that computational times are very important in order to enable real time applications such as autonomous driving. Towards this goal, we present an approach to joint classification, detection and semantic segmentation via a unified architecture where the encoder is shared amongst the three tasks. Our approach is very simple, can be trained end-toend and performs extremely well in the challenging KITTI dataset, outperforming the state-of-the-art in the road segmentation task. Our approach is also very efficient, allowing us to perform inference at more then 23 frames per second.Training scripts and trained weights to reproduce our results can be found here: https://github.com/ MarvinTeichmann/MultiNet

show abstract

Section: Related Workmentioning

confidence: 99%

MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving

Teichmann¹,

Weber

Zöllner

et al. 2018

2018 IEEE Intelligent Vehicles Symposium (IV)

Self Cite

531

339

View full text Add to dashboard Cite

show abstract

“…Brubaker et al [8] developed a technique that can be applied at city scale, without any prior knowledge about the vehicle location. Ma et al [23] incorporated other visual cues, such as the position of the sun and the road type to further improve the results. These works are appealing since they only require a cartographic map.…”

Section: Related Workmentioning

confidence: 99%

“…High-precision Map-based Localization: The proposed work belongs to the category of the high-precision mapbased localization [7,18,19,31,38,39,40,44]. The use of maps has been shown to not only provide strong cues for various tasks in computer vision and robotics such as scene understanding [34], vehicle detection [24], and localization [8,23,35], but also enables the creation of large-scale datasets with little human effort [33,36]. The general idea is to build a centimeter-level high-definition 3D map offline a priori, by stitching sensor input using a high-precision differential GNSS system and offline SLAM.…”

Section: Related Workmentioning

confidence: 99%

“…They also require very large storage on board. Several approaches have been proposed to provide affordable solutions to localization by exploiting coarse maps that are freely available on the web [8,23]. Despite demonstrating promising results, the accuracy of such methods is still in the order of a few meters, which does not meet the requirements of safetycritical applications such as autonomous driving.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localization

Urtasun

Tartavull

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

In this paper we propose a novel semantic localization algorithm that exploits multiple sensors and has precision on the order of a few centimeters. Our approach does not require detailed knowledge about the appearance of the world, and our maps require orders of magnitude less storage than maps utilized by traditional geometry-and LiDAR intensitybased localizers. This is important as self-driving cars need to operate in large environments. Towards this goal, we formulate the problem in a Bayesian filtering framework, and exploit lanes, traffic signs, as well as vehicle dynamics to localize robustly with respect to a sparse semantic map. We validate the effectiveness of our method on a new highway dataset consisting of 312km of roads. Our experiments show that the proposed approach is able to achieve 0.05m lateral accuracy and 1.12m longitudinal accuracy on average while taking up only 0.3% of the storage required by previous LiDAR intensitybased approaches.

show abstract

“…Techniques such as [5], [6] learn end-to-end estimators using deep learning, while others such as [18], [19] combine deep models with traditional estimation machinery. Still other work has focused on training deep models to extract illumination information from images, which can be used to improve the performance of traditional localization systems [20], [21].…”

Section: Related Workmentioning

confidence: 99%

How to Train a CAT: Learning Canonical Appearance Transformations for Direct Visual Localization Under Illumination Change

Clement

Kelly

2018

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Abstract-Direct visual localization has recently enjoyed a resurgence in popularity with the increasing availability of cheap mobile computing power. The competitive accuracy and robustness of these algorithms compared to state-of-the-art feature-based methods, as well as their natural ability to yield dense maps, makes them an appealing choice for a variety of mobile robotics applications. However, direct methods remain brittle in the face of appearance change due to their underlying assumption of photometric consistency, which is commonly violated in practice. In this paper, we propose to mitigate this problem by training deep convolutional encoder-decoder models to transform images of a scene such that they correspond to a previously-seen canonical appearance. We validate our method in multiple environments and illumination conditions using high-fidelity synthetic RGB-D datasets, and integrate the trained models into a direct visual localization pipeline, yielding improvements in visual odometry (VO) accuracy through timevarying illumination conditions, as well as improved metric relocalization performance under illumination change, where conventional methods normally fail. We further provide a preliminary investigation of transfer learning from synthetic to real environments in a localization context.

show abstract

Find your way by observing the sun and other semantic cues

Cited by 41 publications

References 41 publications

MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving

MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving

Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localization

How to Train a CAT: Learning Canonical Appearance Transformations for Direct Visual Localization Under Illumination Change

Contact Info

Product

Resources

About