LLama-SLAM: Learning High-Quality Visual Landmarks for Long-Term Mapping and Localization

Abstract

The precise localization of vehicles is an important requirement for autonomous driving or advanced driver assistance systems. Using common GNSS the ego position can be measured but not with the reliability and precision necessary. An alternative approach to achieve precise localization is the usage of visual landmarks observed by a camera mounted in the vehicle. However, this raises the necessity of reliable visual landmarks that are easily recognizable and persistent. We propose a novel SLAM algorithm that focuses on learning and mapping such visual long-term landmarks (LLamas). The algorithm therefore processes stereo image streams from several recording sessions in the same spatial area. The key part within LLama-SLAM is the assessment of the landmarks with quality values that are inferred as viewpoint dependent probabilities from observation statistics. By adding solely landmarks of high quality to the final LLama Map, it can be kept compact while still allowing reliable localization. Due to the long-term evaluation of the GNSS measurement during the sessions, the landmarks can be positioned precisely in a global referenced coordinate system. For a first assessment of the algorithm's capabilities, we present some experimental results from the mapping process combining three sessions recorded over two months on the same route.

How to Match Tracks of Visual Features for Automotive Long-Term SLAM

“…However, feature matching becomes more challenging in long-term V-SLAMs with landmark observations over several days, weeks and months [1], [2]. Long-term V-SLAMs like LLama-SLAM [9] are advantageous for automotive localization since they offer increased availability, scalability and updatability compared to visual short-term methods. In this paper, we will investigate the feature matching problem in such long-term V-SLAMs.…”

The PRORETA 4 City Assistant System

“…The project was scheduled from 2015 to 2018 and four research assistants from three different institutes of TU Darmstadt worked together on this interdisciplinary project. Within this frame, several articles comprising new algorithms for driver intention detection and online driver adaptation [5][6][7][8][9], visual localization and mapping [10][11][12][13] and driver gaze target estimation [14][15][16][17] have been published as well as articles on safety approval of machine learning algorithms in the automotive context [18]. Many of the core ideas can be retrieved in the exemplary prototypical assistance system that is presented in this work.…”

“…Scenery Model: GNSS fused with speed and acceleration, digital HD-map, see Section 2.1 Camera-based long-term localization with LLama-SLAM, see [11,13]…”

“…This data is provided by a Kalman Filter that fuses GNSS 1 readings with speed and acceleration measurements from the vehicle sensors. To improve the system's capabilities, the ego-vehicle's position could also be determined by a camera-based long-term localization, which was also a research subject in PRORETA 4 [11,13]. All geographical coordinates inside the map as well as vehicle position and orientation are transformed from the common WGS84 coordinate system with latitude and longitude values in an UTM 2 -based coordinate system.…”

“…On the project's website, additional information about the project and the final presentation event can as well be found. The project's research results beyond the City Assistant System are published in [5][6][7][8][9][10][11][12][13][14][15][16][17][18].…”