Fast and incremental loop closure detection with deep features and proximity graphs

Abstract

In recent years, the robotics community has extensively examined methods concerning the place recognition task within the scope of simultaneous localization and mapping applications. This article proposes an appearance‐based loop closure detection pipeline named “Fast and Incremental Loop closure Detection (FILD++). First, the system is fed by consecutive images and, via passing them twice through a single convolutional neural network, global and local deep features are extracted. Subsequently, a hierarchical navigable small‐world graph incrementally constructs a visual database representing the robot's traversed path based on the computed global features. Finally, a query image, grabbed each time step, is set to retrieve similar locations on the traversed route. An image‐to‐image pairing follows, which exploits local features to evaluate the spatial information. Thus, in the proposed article, we propose a single network for global and local feature extraction in contrast to our previous work (FILD), while an exhaustive search for the verification process is adopted over the generated deep local features avoiding the utilization of hash codes. Exhaustive experiments on eleven publicly available data sets exhibit the system's high performance (achieving the highest recall score on eight of them) and low execution times (22.05 ms on average in New College, which is the largest one containing 52,480 images) compared to other state‐of‐the‐art approaches.

Tracking‐DOSeqSLAM: A dynamic sequence‐based visual place recognition paradigm

“…The term N/A denotes that the corresponding information is not available from any cited source. Furthermore, for the case of FAB‐MAP 2.0 [38] and DBoW2 [40] along with FILD [52] where no actual measurements are provided regarding the used datasets, the presented results are obtained from the setup described in [55, 81], respectively. Most of the approaches (e.g.…”

“…-259 represent an image and determine potentially revisited locations [46][47][48][49][50][51][52][53][54][55][56]. These layers are originally trained for object recognition; thus, they are tightly bounded to their learning example attributes.…”

“…The latest works in visual loop closure detection are inspired by the great success of CNNs in several computer vision tasks [43–45]. These approaches address the place recognition task by using specific layers of the architecture to represent an image and determine potentially revisited locations [46–56]. These layers are originally trained for object recognition; thus, they are tightly bounded to their learning example attributes.…”

Automatic vocabulary and graph verification for accurate loop closure detection

“…Recently, Ma et al (2021) proposed a tailormade feature matching method based on locality consistency to check the geometric relationship of detections. As a de-facto standard post-verification procedure, scholars widely apply epipolar check which estimates the fundamental matrix between features across images based on the random sample consensus (RANSAC) (Fischler & Bolles, 1981) procedure and then validates the detection by the number of feature inliers (An et al, 2022;Galvez-López & Tardos, 2012;Garcia-Fidalgo & Ortiz, 2017;Galvez-López & Tardós, 2011;Lynen et al, 2017;Tsintotas et al, 2018Tsintotas et al, , 2019. It works properly in most scenes but fails when estimation is unreliable due to dominated outliers or incorrect parametric model.…”

“…In recent years, motivated by the success of deep CNN in other computer vision tasks, many novel learned features (Arandjelovi et al, 2018; DeTone et al, 2018; Dusmanu et al, 2019; Noh et al, 2017; Sarlin et al, 2019) have been proposed and shown improved robustness against varying illuminations and viewpoints than traditional counterparts. Some scholars have begun to use CNN‐based features to perform loop detection, such as NetVLAD (Arandjelovi et al, 2018), DELF (Noh et al, 2017), HF‐Net (Sarlin et al, 2019), and other CNN‐based features (An et al, 2022; Hou et al, 2018; Kenshimov et al, 2017; Lopez‐Antequera et al, 2017; Memon et al, 2020; Yue et al, 2019).…”