Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition

Hausler, Stephen; Garg, Sourav; Mei, Xu; Milford, Michael; Fischer, Tobias

doi:10.1109/cvpr46437.2021.01392

Cited by 223 publications

(199 citation statements)

References 79 publications

Supporting

Mentioning

196

Contrasting

Order By: Relevance

“…MSW performed better than object proposal techniques due to the use of the sliding window, especially when illumination or viewpoint changes. Similar to MSW, Patch-NetVLAD [10] also used the sliding window to generate multi-scale patches and obtained patch descriptors from NetVLAD residuals. However, these patches are described using NetVLAD trained on the whole image, which is not accurate.…”

Section: Descriptor-centered Methodsmentioning

confidence: 99%

“…These global descriptors [1,6,7] which describe the whole image typically excel in terms of their robustness to appearance and illumination changes, as they are directly optimized for place recognition. However, as shown in [10,23,29], in areas with similar scenes, the global descriptor has difficulty distinguishing differences between local regions.…”

Section: Introductionmentioning

confidence: 99%

“…We propose to assign weights to patches based on the distance between patch descriptors and centroids in the description space so that patches in LSR play an important role in image matching. [10], which uses a sliding window to generate patches and then derives patch descriptors from NetVLAD to achieve place recognition. However, Patch-NetVLAD extracted patch descriptors using NetVLAD trained on the whole image, which is not accurate.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Patch-NetVLAD+: Learned patch descriptor and weighted matching strategy for place recognition

Cai¹,

Zhao²,

Cui³

et al. 2022

Preprint

View full text Add to dashboard Cite

Visual Place Recognition (VPR) in areas with similar scenes such as urban or indoor scenarios is a major challenge. Existing VPR methods using global descriptors have difficulty capturing local specific regions (LSR) in the scene and are therefore prone to localization confusion in such scenarios. As a result, finding the LSR that are critical for location recognition becomes key. To address this challenge, we introduced Patch-NetVLAD+, which was inspired by patch-based VPR researches. Our method proposed a fine-tuning strategy with triplet loss to make NetVLAD suitable for extracting patch-level descriptors. Moreover, unlike existing methods that treat all patches in an image equally, our method extracts patches of LSR, which present less frequently throughout the dataset, and makes them play an important role in VPR by assigning proper weights to them. Experiments on Pittsburgh30k and Tokyo247 datasets show that our approach achieved up to 6.35% performance improvement than existing patch-based methods.

show abstract

Section: Descriptor-centered Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Patch-NetVLAD+: Learned patch descriptor and weighted matching strategy for place recognition

Cai¹,

Zhao²,

Cui³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…One focus within VPR research is around designing methods that can successfully match places across changing appearance conditions such as day/night and seasonal changes [1], [4], [8], as well as across viewpoint changes [21], [25]. VPR approaches can be further partitioned into methods that perform matching using deep learning [4], [5], [8], [26] (see [27] for a review) or handcrafted visual features [1], [3] (see [28] for a review). Common across many of these approaches is a match scoring system; given a pair of images, a typical VPR system outputs a "quality score" which indicates the likelihood of two images being captured from the same place.…”

Section: A Visual Place Recognitionmentioning

confidence: 99%

Probabilistic Appearance-Invariant Topometric Localization With New Place Awareness

Mei

Fischer

Sünderhauf

et al. 2021

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

Probabilistic state-estimation approaches offer a principled foundation for designing localization systems, because they naturally integrate sequences of imperfect motion and exteroceptive sensor data. Recently, probabilistic localization systems utilizing appearance-invariant visual place recognition (VPR) methods as the primary exteroceptive sensor have demonstrated state-of-the-art performance in the presence of substantial appearance change. However, existing systems 1) do not fully utilize odometry data within the motion models, and 2) are unable to handle route deviations, due to the assumption that query traverses exactly repeat the mapping traverse. To address these shortcomings, we present a new probabilistic topometric localization system which incorporates full 3-dof odometry into the motion model and furthermore, adds an "off-map" state within the state-estimation framework, allowing query traverses which feature significant route detours from the reference map to be successfully localized. We perform extensive evaluation on multiple query traverses from the Oxford RobotCar dataset exhibiting both significant appearance change and deviations from routes previously traversed. In particular, we evaluate performance on two practically relevant localization tasks: loop closure detection and global localization. Our approach achieves major performance improvements over both existing and improved state-of-the-art systems.

show abstract

“…More recently, NetVLAD [4] first proposed an end-to-end place recognition network by using a CNN for feature extraction and a differentiable NetVLAD layer for global aggregation. This inspired a series of end-to-end visual place recognition networks [5], [6], [7], [8], [9]. To utilise 3-D information, PointNetVLAD [10] and its successors [11], [12], [13], [14], [15] use point clouds as inputs and achieve very high average recall rates in outdoor environments with 3-D features aggregated Fig.…”

Section: Introductionmentioning

confidence: 99%

CGS-Net: Aggregating Colour, Geometry and Semantic Features for Large-Scale Indoor Place Recognition

Ming¹,

Yang²,

Calway³

2022

Preprint

View full text Add to dashboard Cite

We describe an approach to large-scale indoor place recognition that aggregates low-level colour and geometric features with high-level semantic features. We use a deep learning network that takes in RGB point clouds and extracts local features with five 3-D kernel point convolutional (KPConv) layers. We specifically train the KPConv layers on the semantic segmentation task to ensure that the extracted local features are semantically meaningful. Then, feature maps from all the five KPConv layers are concatenated together and fed into the NetVLAD layer to generate the global descriptors. The approach is trained and evaluated using a large-scale indoor place recognition dataset derived from the ScanNet dataset, with a test set comprising 3,608 point clouds generated from 100 different rooms. Comparison with a traditional feature based method and three state-of-the-art deep learning methods demonstrate that the approach significantly outperforms all four methods, achieving, for example, a top-3 average recall rate of 75% compared with 41% for the closest rival method.

show abstract

Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition

Cited by 223 publications

References 79 publications

Patch-NetVLAD+: Learned patch descriptor and weighted matching strategy for place recognition

Patch-NetVLAD+: Learned patch descriptor and weighted matching strategy for place recognition

Probabilistic Appearance-Invariant Topometric Localization With New Place Awareness

CGS-Net: Aggregating Colour, Geometry and Semantic Features for Large-Scale Indoor Place Recognition

Contact Info

Product

Resources

About