DASGIL: Domain Adaptation for Semantic and Geometric-Aware Image-Based Localization

Learning Semantics for Visual Place Recognition through Multi-Scale Attention

“…We assess the effectiveness of our approach comparing it to GeM [21] and RMAC [25], two state-of-the-art VPR methods that use global descriptors. We also compare to DASGIL [11] which, similarly to our method, uses semantic information albeit selecting it in a top-down manner. Besides semantics, DASGIL can also leverage depth information to further enhance the descriptors.…”

“…Along these lines, the method presented in [19] requires segmenting an images also at inference time while [15] requires a 3D point cloud of the scene. Closely related to our work is DASGIL [11], an architecture that uses a single encoder shared by three tasks (VPR, depth mask reconstruction and semantic mask reconstruction) to create embeddings that fuse visual, geometric and semantic information. Similar to our solution, DASGIL is trained on a synthetic dataset and it uses domain adaptation to align the features extracted from the synthetic and real-world domains.…”

“…Despite the advances in this direction made possible by the use of deep convolutional neural networks (DCNN) [2,21,25], current VPR solutions still fail to achieve the degree of generality and flexibility required to work across different environments and conditions [31]. Recent studies have found that these problems can be mitigated by building image descriptors based not only on visual appearance, but also on the semantic content in the scene [19,15,11]. Intuitively, dynamic objects or elements that are both common across all places and that lack distinctive features (e.g.…”

“…appropriate annotated database from where to learn it. Indeed, previous works on combining appearance and semantic information to generate global descriptors for VPR have either empirically defined what semantic classes should be used to describe places [18,19] or used all the semantic content available [11]. While this can be seen as a legitimate choice, it remains that it does not exist a public VPR dataset containing images annotated with pixel-wise semantic maps.…”

Efficient 3D Deep LiDAR Odometry

“…Recently, learning-based odometry methods have shown impressive accuracy on datasets compared with conventional methods based on hand-crafted features. It is found that learning-based methods can deal with sparse features and dynamic environments [1], [2], which are usually difficult for conventional methods. To our knowledge, most learning-based methods are on the 2D visual odometry [3], [4], [5], [6], [7], [8], [9] or utilize 2D convolution on the projected information of LiDAR [10], [11], [12], [13], [14].…”