FusionVLAD: A Multi-View Deep Fusion Networks for Viewpoint-Free 3D Place Recognition

Yin, Peng; Xu, Lingyun; Zhang, Ji; Choset, Howie

doi:10.1109/lra.2021.3061375

Cited by 23 publications

(16 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our method also directly generates global descriptors on LiDAR scans. Different from methods that use local point cloud maps [18], [24], [33], [34], our method only uses range images generated from single 3D LiDAR scans. This yields fast computations suitable for online operation.…”

Section: Related Workmentioning

confidence: 99%

OverlapTransformer: An Efficient and Rotation-Invariant Transformer Network for LiDAR-Based Place Recognition

Ma¹,

Zhang²,

Xu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Place recognition is an important capability for autonomously navigating vehicles operating in complex environments and under changing conditions. It is a key component for tasks such as loop closing in SLAM or global localization. In this paper, we address the problem of place recognition based on 3D LiDAR scans recorded by an autonomous vehicle. We propose a novel lightweight neural network exploiting the range image representation of LiDAR sensors to achieve fast execution with less than 4 ms per frame. Based on that, we design a yaw-rotation-invariant architecture exploiting a transformer network, which boosts the place recognition performance of our method. We evaluate our approach on the KITTI and Ford Campus datasets. The experimental results show that our method can effectively detect loop closures compared to the state-of-the-art methods and generalizes well across different environments. To further evaluate long-term place recognition performance, we provide a novel challenging Haomo dataset, which contains LiDAR sequences recorded by a mobile robot in repetitive places across seasons. Both the implementation of our method and our new Haomo dataset are released here: https://github.com/haomo-ai/OverlapTransformer

show abstract

Section: Related Workmentioning

confidence: 99%

OverlapTransformer: An Efficient and Rotation-Invariant Transformer Network for LiDAR-Based Place Recognition

Ma¹,

Zhang²,

Xu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, PointNetVLAD [5] combines PointNet and NetVLAD [4] to enable an end-to-end training and extraction for global descriptor from 3D point clouds. Others like LPD-Net [7] and FusionVLAD [8] further deal with the feature aggregation and the viewpoint difference problems, respectively. Similar to vision-based methods, some researchers have applied the attention mechanism to networks for better concentration on important features, such as PCAN [16], SOE-Net [17] and AttDLNet [18].…”

Section: Related Workmentioning

confidence: 99%

“…contrast, LiDARs are more robust to the variation of light and time [5], and several LiDAR-based methods [5]- [8] have been developed for long-term localization. Despite the precise geometric structure information from point clouds, they may encounter failure in some degenerate places like corridors and tunnels.…”

mentioning

confidence: 99%

AdaFusion: Visual-LiDAR Fusion with Adaptive Weights for Place Recognition

Lai¹,

Yin²,

Scherer³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Recent years have witnessed the increasing application of place recognition in various environments, such as city roads, large buildings, and a mix of indoor and outdoor places. This task, however, still remains challenging due to the limitations of different sensors and the changing appearance of environments. Current works only consider the use of individual sensors, or simply combine different sensors, ignoring the fact that the importance of different sensors varies as the environment changes. In this paper, an adaptive weighting visual-LiDAR fusion method, named AdaFusion, is proposed to learn the weights for both images and point cloud features. Features of these two modalities are thus contributed differently according to the current environmental situation. The learning of weights is achieved by the attention branch of the network, which is then fused with the multi-modality feature extraction branch. Furthermore, to better utilize the potential relationship between images and point clouds, we design a twostage fusion approach to combine the 2D and 3D attention. Our work is tested on two public datasets, and experiments show that the adaptive weights help improve recognition accuracy and system robustness to varying environments.

show abstract

“…This inspired a series of end-to-end visual place recognition networks [5], [6], [7], [8], [9]. To utilise 3-D information, PointNetVLAD [10] and its successors [11], [12], [13], [14], [15] use point clouds as inputs and achieve very high average recall rates in outdoor environments with 3-D features aggregated Fig. 1.…”

Section: Introductionmentioning

confidence: 99%

CGS-Net: Aggregating Colour, Geometry and Semantic Features for Large-Scale Indoor Place Recognition

Ming¹,

Yang²,

Calway³

2022

Preprint

View full text Add to dashboard Cite

We describe an approach to large-scale indoor place recognition that aggregates low-level colour and geometric features with high-level semantic features. We use a deep learning network that takes in RGB point clouds and extracts local features with five 3-D kernel point convolutional (KPConv) layers. We specifically train the KPConv layers on the semantic segmentation task to ensure that the extracted local features are semantically meaningful. Then, feature maps from all the five KPConv layers are concatenated together and fed into the NetVLAD layer to generate the global descriptors. The approach is trained and evaluated using a large-scale indoor place recognition dataset derived from the ScanNet dataset, with a test set comprising 3,608 point clouds generated from 100 different rooms. Comparison with a traditional feature based method and three state-of-the-art deep learning methods demonstrate that the approach significantly outperforms all four methods, achieving, for example, a top-3 average recall rate of 75% compared with 41% for the closest rival method.

show abstract

FusionVLAD: A Multi-View Deep Fusion Networks for Viewpoint-Free 3D Place Recognition

Cited by 23 publications

References 17 publications

OverlapTransformer: An Efficient and Rotation-Invariant Transformer Network for LiDAR-Based Place Recognition

OverlapTransformer: An Efficient and Rotation-Invariant Transformer Network for LiDAR-Based Place Recognition

AdaFusion: Visual-LiDAR Fusion with Adaptive Weights for Place Recognition

CGS-Net: Aggregating Colour, Geometry and Semantic Features for Large-Scale Indoor Place Recognition

Contact Info

Product

Resources

About