Real-Time Visual Place Recognition Based on Analyzing Distribution of Multi-scale CNN Landmarks

Xin, Zhe; Cui, Xiaoguang; Zhang, Jixiang; Yang, Yiping; Wang, Yanqing

doi:10.1007/s10846-018-0804-x

Cited by 17 publications

(8 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More recent approaches include ranking-loss based learning [56], novel pooling [55], contextual feature reweighting [37], large scale re-training [79], semantics-guided feature aggregation [25,61,72], use of 3D [50,78,40], additional sensors [29,52,22] and image appearance translation [1,54]. Place matches obtained through global descriptor matching are often re-ranked using sequential information [24,82,46], query expansion [28,13], geometric verification [38,25,49] and feature fusion [80,83]. Distinct from existing approaches, this paper introduces Patch-NetVLAD, which reverses the local-to-global process of image description by deriving multi-scale patch features from a global descriptor, NetVLAD.…”

Section: Related Workmentioning

confidence: 99%

“…Existing techniques for multi-scale approaches typically fuse information at the descriptor level, which can lead to loss of complementary or discriminative cues [80,83,10,48,27,87] due to pooling, or increased descriptor sizes due to concatenation [39,86,7,8]. Distinct from these methods, we consider multi-scale fusion at the final scoring stage, which enables parallel processing with associated speed benefits.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition

Hausler¹,

Garg²,

Xu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Visual Place Recognition is a challenging task for robotics and autonomous systems, which must deal with the twin problems of appearance and viewpoint change in an always changing world. This paper introduces Patch-NetVLAD, which provides a novel formulation for combining the advantages of both local and global descriptor methods by deriving patch-level features from NetVLAD residuals. Unlike the fixed spatial neighborhood regime of existing local keypoint features, our method enables aggregation and matching of deep-learned local features defined over the feature-space grid. We further introduce a multi-scale fusion of patch features that have complementary scales (i.e. patch sizes) via an integral feature space and show that the fused features are highly invariant to both condition (season, structure, and illumination) and viewpoint (translation and rotation) changes. Patch-NetVLAD outperforms both global and local feature descriptor-based methods with comparable compute, achieving state-of-the-art visual place recognition results on a range of challenging real-world datasets, including winning the Facebook Mapillary Visual Place Recognition Challenge at ECCV2020. It is also adaptable to user requirements, with a speed-optimised version operating over an order of magnitude faster than the stateof-the-art. By combining superior performance with improved computational efficiency in a configurable framework, Patch-NetVLAD is well suited to enhance both stand-alone place recognition capabilities and the overall performance of SLAM systems.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition

Hausler¹,

Garg²,

Xu³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Considering that CNN leads to the large space of image representation, Chen et al 30 proposed a new method of reserving salient feature maps and make adaptive binarization on it, and the experimental results proved the effectiveness. Since the highly representative landmark features are robust to appearance changes, Xin et al 31 proposed an effective method based on CNNs and content-based multiscale landmarks to complete the task of place recognition. Chen et al 32 proposed a method for place recognition based on CNN features, which achieved the aim of recognizing the place.…”

Section: Related Workmentioning

confidence: 99%

Robust place recognition based on salient landmarks screening and convolutional neural network features

Niu

Qian

2020

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

In this work, we propose a robust place recognition measurement in natural environments based on salient landmark screening and convolutional neural network (CNN) features. First, the salient objects in the image are segmented as candidate landmarks. Then, a category screening network is designed to remove specific object types that are not suitable for environmental modeling. Finally, a three-layer CNN is used to get highly representative features of the salient landmarks. In the similarity measurement, a Siamese network is chosen to calculate the similarity between images. Experiments were conducted on three challenging benchmark place recognition datasets and superior performance was achieved compared to other state-of-the-art methods, including FABMAP, SeqSLAM, SeqCNNSLAM, and PlaceCNN. Our method obtains the best results on the precision–recall curves, and the average precision reaches 78.43%, which is the best of the comparison methods. This demonstrates that the CNN features on the screened salient landmarks can be against a strong viewpoint and condition variations.

show abstract

“…The authors in [68] proposed an attention mechanism capable of being incorporated into an existing feed-forward network architecture in order to learn image representations for longterm place recognition applications. An effective similarity measurement for the detection of pre-visited locations in changing environments was proposed in [69]. Combining a neural network inspired by the Drosophila olfactory neural circuit (FlyNet) and a 1-d Continuous Attractor Neural Network (CANN), a compact system with high performances was proposed by [70].…”

Section: B Approaches Using Convolutional Neural Network Featuresmentioning

confidence: 99%

Fast and Incremental Loop Closure Detection Using Proximity Graphs

Che

Zhou

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Visual loop closure detection, which can be considered as an image retrieval task, is an important problem in SLAM (Simultaneous Localization and Mapping) systems. The frequently used bag-of-words (BoW) models can achieve high precision and moderate recall. However, the requirement for lower time costs and fewer memory costs for mobile robot applications is not well satisfied. In this paper, we propose a novel loop closure detection framework titled 'FILD' (Fast and Incremental Loop closure Detection), which focuses on an on-line and incremental graph vocabulary construction for fast loop closure detection. The global and local features of frames are extracted using the Convolutional Neural Networks (CNN) and SURF on the GPU, which guarantee extremely fast extraction speeds. The graph vocabulary construction is based on one type of proximity graph, named Hierarchical Navigable Small World (HNSW) graphs, which is modified to adapt to this specific application. In addition, this process is coupled with a novel strategy for real-time geometrical verification, which only keeps binary hash codes and significantly saves on memory usage. Extensive experiments on several publicly available datasets show that the proposed approach can achieve fairly good recall at 100% precision compared to other stateof-the-art methods. The source code can be downloaded at https://github.com/AnshanTJU/FILD for further studies.

show abstract

Real-Time Visual Place Recognition Based on Analyzing Distribution of Multi-scale CNN Landmarks

Cited by 17 publications

References 25 publications

Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition

Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition

Robust place recognition based on salient landmarks screening and convolutional neural network features

Fast and Incremental Loop Closure Detection Using Proximity Graphs

Contact Info

Product

Resources

About