AdaFusion: Visual-LiDAR Fusion With Adaptive Weights for Place Recognition

Lai, Haowen; Yin, Peng; Scherer, Sebastian

doi:10.1109/lra.2022.3210880

Cited by 23 publications

(7 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recall@1 Recall@5 Recall@10 max F In the following experiments, we compare the performance of our LCPR with the state-of-the-art baselines including the vision-based method NetVLAD [6], the radarbased method AutoPlace [38], the LiDAR-based methods PointNetVLAD [10] and OverlapTransformer [2], and multimodal methods including MinkLoc++ [13], PIC-Net [14], and AdaFusion [12]. We use the released open-source codes of the baseline methods for evaluation except for PIC-Net [14], which is implemented ourselves according to their original paper.…”

Section: Methodsmentioning

confidence: 99%

“…Pointwise image features are first extracted using the method described in [34], then point cloud features and pointwise image features are aggregated by a convolution layer. To better exploit the discriminativeness of the descriptors from different modalities, Lai et al [12] (AdaFusion) design an attention branch in their network to weight modalities adaptively.…”

Section: Fusion-based Place Recognitionmentioning

confidence: 99%

“…The combination of LiDARs and cameras can surpass the individual limitations within each modality, which is widely used in autonomous driving perception tasks, such as object detection, semantic segmentation, and depth prediction. In the field of place recognition, some multimodal fusion-based approaches have been proposed, such as AdaFusion [12], Min-kLoc++ [13], PIC-Net [14], proving that multimodal fusion can enhance the performance of recognition. However, the limited field of view (FOV) and viewpoint of the image data used by these methods introduce an inherent flaw when the observation angles are altered by mounting positions or driving directions, resulting in an imbalance between features from different modalities for sensor fusion.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An attention-based multi-scale convolution network for intelligent underwater acoustic signal recognition

Zhou,

Li,

Zhang

et al. 2023

Ocean Engineering

View full text Add to dashboard Cite

Section: Methodsmentioning

confidence: 99%

Section: Fusion-based Place Recognitionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An attention-based multi-scale convolution network for intelligent underwater acoustic signal recognition

Zhou,

Li,

Zhang

et al. 2023

Ocean Engineering

View full text Add to dashboard Cite

“…The final global descriptor is concatenated with the 2D image descriptor and 3D point cloud descriptor. AdaFusion (Lai, Yin, and Scherer 2022) leverages a multi-scale attention module that hierarchically aggregates multi-modal features.…”

Section: Visual Place Recognitionmentioning

confidence: 99%

DistilVPR: Cross-Modal Knowledge Distillation for Visual Place Recognition

Wang,

She,

Kang

et al. 2024

AAAI

View full text Add to dashboard Cite

The utilization of multi-modal sensor data in visual place recognition (VPR) has demonstrated enhanced performance compared to single-modal counterparts. Nonetheless, integrating additional sensors comes with elevated costs and may not be feasible for systems that demand lightweight operation, thereby impacting the practical deployment of VPR. To address this issue, we resort to knowledge distillation, which empowers single-modal students to learn from cross-modal teachers without introducing additional sensors during inference. Despite the notable advancements achieved by current distillation approaches, the exploration of feature relationships remains an under-explored area. In order to tackle the challenge of cross-modal distillation in VPR, we present DistilVPR, a novel distillation pipeline for VPR. We propose leveraging feature relationships from multiple agents, including self-agents and cross-agents for teacher and student neural networks. Furthermore, we integrate various manifolds, characterized by different space curvatures for exploring feature relationships. This approach enhances the diversity of feature relationships, including Euclidean, spherical, and hyperbolic relationship modules, thereby enhancing the overall representational capacity. The experiments demonstrate that our proposed pipeline achieves state-of-the-art performance compared to other distillation baselines. We also conduct necessary ablation studies to show design effectiveness. The code is released at: https://github.com/sijieaaa/DistilVPR

show abstract

“…Place recognition provides the current global location of the vehicle in the previously seen environments, which is an important component of robotic simultaneous localization and mapping (SLAM) and global localization. During online operation, it retrieves the reference scan in the database most similar to the current query by directly regressing the similarity [1], [2] or descriptor matching [3], [4], [5]. LiDARbased place recognition methods [6], [7], [8] can be applied to large-scale outdoor environments due to their robustness to illumination and weather changes.…”

Section: Introductionmentioning

confidence: 99%

CVTNet: A Cross-View Transformer Network for Place Recognition Using LiDAR Data

Ma¹,

Guan²,

Xu³

et al. 2023

Preprint

View full text Add to dashboard Cite

LiDAR-based place recognition (LPR) is one of the most crucial components of autonomous vehicles to identify previously visited places in GPS-denied environments. Most existing LPR methods use mundane representations of the input point cloud without considering different views, which may not fully exploit the information from LiDAR sensors. In this paper, we propose a cross-view transformer-based network, dubbed CVTNet, to fuse the range image views (RIVs) and bird's eye views (BEVs) generated from the LiDAR data. It extracts correlations within the views themselves using intra-transformers and between the two different views using inter-transformers.Based on that, our proposed CVTNet generates a yaw-angleinvariant global descriptor for each laser scan end-to-end online and retrieves previously seen places by descriptor matching between the current query scan and the pre-built database. We evaluate our approach on three datasets collected with different sensor setups and environmental conditions. The experimental results show that our method outperforms the state-of-the-art LPR methods with strong robustness to viewpoint changes and long-time spans. Furthermore, our approach has a good realtime performance that can run faster than the typical LiDAR frame rate. The implementation of our method is released as open source at: https://github.com/BIT-MJY/CVTNet.

show abstract

AdaFusion: Visual-LiDAR Fusion With Adaptive Weights for Place Recognition

Cited by 23 publications

References 29 publications

An attention-based multi-scale convolution network for intelligent underwater acoustic signal recognition

An attention-based multi-scale convolution network for intelligent underwater acoustic signal recognition

DistilVPR: Cross-Modal Knowledge Distillation for Visual Place Recognition

CVTNet: A Cross-View Transformer Network for Place Recognition Using LiDAR Data

Contact Info

Product

Resources

About