VLASE: Vehicle Localization by Aggregating Semantic Edges

Yu, Xin; Chaturvedi, Sagar; Feng, Chen; Taguchi, Yuichi; Lee, Teng-Yok; Fernandes, Clinton; Ramalingam, Srikumar

doi:10.1109/iros.2018.8594358

Cited by 43 publications

(29 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Visual localization is the problem of estimating the camera pose of an image [8,34], typically from a set of 2D-3D matches between image pixels and 3D scene points. In the context of long-term visual localization [58,[63][64][65]74], semantics are proven useful to be able to handle variations in scene geometry and appearance, e.g., due to seasonal changes. These methods are based on the idea that the semantic meaning of a scene part is invariant to such changes.…”

Section: Introductionmentioning

confidence: 99%

A Cross-Season Correspondence Dataset for Robust Semantic Segmentation

Larsson

Stenborg

Hammarstrand

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

In this paper, we present a method to utilize 2D-2D point matches between images taken during different image conditions to train a convolutional neural network for semantic segmentation. Enforcing label consistency across the matches makes the final segmentation algorithm robust to seasonal changes. We describe how these 2D-2D matches can be generated with little human interaction by geometrically matching points from 3D models built from images. Two cross-season correspondence datasets are created providing 2D-2D matches across seasonal changes as well as from day to night. The datasets are made publicly available to facilitate further research. We show that adding the correspondences as extra supervision during training improves the segmentation performance of the convolutional neural network, making it more robust to seasonal changes and weather conditions.

show abstract

Section: Introductionmentioning

confidence: 99%

A Cross-Season Correspondence Dataset for Robust Semantic Segmentation

Larsson

Stenborg

Hammarstrand

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…Besides using learned features that are more robust to changes in viewing conditions, long-term localization approaches also use semantic image segmentation (Budvytis et al 2019;Garg et al 2019;Larsson et al 2019;Stenborg et al 2018;Schönberger et al 2018;Seymour et al 2019;Shi et al 2019;Taira et al 2019;Toft et al 2017Toft et al , 2018Wang et al 2019;Yu et al 2018). These methods are based on the observation that the semantic meaning of scene elements, in contrast to their appearance, is invariant to changes.…”

Section: Semantic Visual Localizationmentioning

confidence: 99%

“…These methods are based on the observation that the semantic meaning of scene elements, in contrast to their appearance, is invariant to changes. Semantic image segmentations are thus used as an invariant representation for image retrieval (Arandjelović and Zisserman 2014;Toft et al 2017;Yu et al 2018), to verify 2D-3D matches (Budvytis et al 2019;Larsson et al 2019;Stenborg et al 2018;Toft et al 2018) and camera pose estimates (Shi et al 2019;Stenborg et al 2018;Taira et al 2019;Toft et al 2018), for learning local features (Garg et al 2019;Schönberger et al 2018), and as an additional input to learning-based localization approaches (Budvytis et al 2019;Seymour et al 2019;Wang et al 2019).…”

Section: Semantic Visual Localizationmentioning

confidence: 99%

Reference Pose Generation for Long-term Visual Localization via Learned Features and View Synthesis

2020

View full text Add to dashboard Cite

Visual Localization is one of the key enabling technologies for autonomous driving and augmented reality. High quality datasets with accurate 6 Degree-of-Freedom (DoF) reference poses are the foundation for benchmarking and improving existing methods. Traditionally, reference poses have been obtained via Structure-from-Motion (SfM). However, SfM itself relies on local features which are prone to fail when images were taken under different conditions, e.g., day/night changes. At the same time, manually annotating feature correspondences is not scalable and potentially inaccurate. In this work, we propose a semi-automated approach to generate reference poses based on feature matching between renderings of a 3D model and real images via learned features. Given an initial pose estimate, our approach iteratively refines the pose based on feature matches against a rendering of the model from the current pose estimate. We significantly improve the nighttime reference poses of the popular Aachen Day–Night dataset, showing that state-of-the-art visual localization methods perform better (up to 47%) than predicted by the original reference poses. We extend the dataset with new nighttime test images, provide uncertainty estimates for our new reference poses, and introduce a new evaluation criterion. We will make our reference poses and our framework publicly available upon publication.

show abstract

“…[3] uses object detection and random finite set representations to localize against a map of semantic objects. Semantic segmentations are for example used in [69] to evaluate the consistency of feature matches for localization, in [63] for geo-location including discovery of commonly occurring scene layouts, and in [75] to obtain semantic edges that can be used for localization. [17] extract semantic graphs from semantic segmentation and use them to compute random walk descriptors for fast matching against a database graph.…”

Section: B Exploiting Semantic For Localizationmentioning

confidence: 99%

Vector Semantic Representations as Descriptors for Visual Place Recognition

Neubert¹,

Schubert²,

Schlegel³

et al. 2021

Robotics: Science and Systems XVII

View full text Add to dashboard Cite

Place recognition is the task of recognizing the current scene from a database of known places. The currently dominant algorithmic paradigm is to use (deep learning based) holistic feature vectors to describe each place and use fast vector query methods to find matchings. We propose a novel type of image descriptor, Vector Semantic Representations (VSR), that encodes the spatial semantic layout from a semantic segmentation together with appearance properties in a, for example, 4,096 dimensional vector for place recognition. We leverage operations from the established class of Vector Symbolic Architectures to combine symbolic (e.g. class label) and numeric (e.g. feature map response) information in a common vector representation. We evaluate the proposed semantic descriptor on 13 standard mobile robotic place recognition datasets and compare to six descriptors from the literature. VSR is on par with the best compared descriptor (NetVLAD) in terms of mean average precision and superior in terms of recall and worst-case average precision. This makes the approach particularly interesting for candidate selection. For a more detailed investigation, we discuss and evaluate recall integrity as additional criterion. Further, we demonstrate that the semantic descriptor is particularly well suited for combination with existing appearance descriptors indicating that semantics provide complementary information for image matching.

show abstract

VLASE: Vehicle Localization by Aggregating Semantic Edges

Cited by 43 publications

References 55 publications

A Cross-Season Correspondence Dataset for Robust Semantic Segmentation

A Cross-Season Correspondence Dataset for Robust Semantic Segmentation

Reference Pose Generation for Long-term Visual Localization via Learned Features and View Synthesis

Vector Semantic Representations as Descriptors for Visual Place Recognition

Contact Info

Product

Resources

About