Sequence searching with CNN features for robust and fast visual place recognition

Bai, Ding; Wang, Chaoqun; Zhang, Bo; Yi, Xiaodong; Yang, Xuejun

doi:10.1016/j.cag.2017.07.019

Cited by 37 publications

(34 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• SeqSLAM [3]: State-of-the-art condition-invariant place recognition method. The off-the-shelf deep-learned feature representations, especially 'conv-3', have been used for place recognition by authors in [36], [37], [15], [22], [7].…”

Section: Performance Comparisonmentioning

confidence: 99%

Don't Look Back: Robustifying Place Categorization for Viewpoint- and Condition-Invariant Place Recognition

Garg

Suenderhauf

Milford

2018

2018 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

When a human drives a car along a road for the first time, they later recognize where they are on the return journey typically without needing to look in their rear view mirror or turn around to look back, despite significant viewpoint and appearance change. Such navigation capabilities are typically attributed to our semantic visual understanding of the environment [1] beyond geometry to recognizing the types of places we are passing through such as "passing a shop on the left" or "moving through a forested area". Humans are in effect using place categorization [2] to perform specific place recognition even when the viewpoint is 180 degrees reversed. Recent advances in deep neural networks have enabled high performance semantic understanding of visual places and scenes, opening up the possibility of emulating what humans do. In this work, we develop a novel methodology for using the semantics-aware higher-order layers of deep neural networks for recognizing specific places from within a reference database. To further improve the robustness to appearance change, we develop a descriptor normalization scheme that builds on the success of normalization schemes for pure appearance-based techniques such as SeqSLAM [3]. Using two different datasets -one road-based, one pedestrian-based, we evaluate the performance of the system in performing place recognition on reverse traversals of a route with a limited field of view camera and no turn-back-and-look behaviours, and compare to existing stateof-the-art techniques and vanilla off-the-shelf features. The results demonstrate significant improvements over the existing state of the art, especially for extreme perceptual challenges that involve both great viewpoint change and environmental appearance change. We also provide experimental analyses of the contributions of the various system components: the use of spatio-temporal sequences, place categorization and placecentric characteristics as opposed to object-centric semantics.

show abstract

Section: Performance Comparisonmentioning

confidence: 99%

Don't Look Back: Robustifying Place Categorization for Viewpoint- and Condition-Invariant Place Recognition

Garg

Suenderhauf

Milford

2018

2018 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

show abstract

“…Kenshimov et al [32] proposed a method to omit parts of the activation maps from the neural networks in order to improve cross-seasonal place recognition. Hou et al [14] combined ConvNet features with a bag of words scheme to speed up querying, while Bai et al [33] combined ConvNet features with sequence searching to increase reliability. All of these methods rely on features extracted from generic neural networks that are not trained specifically for loop closure.…”

Section: Related Workmentioning

confidence: 99%

“…We seek to emphasize that our method of creating and querying the database with the descriptors extracted from our model is simple but effective; albeit, we are able to achieve faster-than-real-time querying speed with minimal memory usage (see Section IV-E). Furthermore, since many new ConvNet-based place recognition systems [12,32,14,33] rely on features from bulky off-the-shelf networks, our lightweight model can potentially be utilized in many of these systems to achieve speedups with competative accuracy (see Section IV-G).…”

Section: Online Usementioning

confidence: 99%

“…Additionally, we choose AlexNet conv3 features both with GRP, compressed to 1,064 dimensions, and in original form, since it is a popular choice for ConvNet-based place recognition (i.e. in [12,14,33,32]). Note that AlexNet has been modified to only contain up to the conv3 layer here for fair testing.…”

Section: E Runtime Evaluationmentioning

confidence: 99%

“…As stated in Section III-D, our model can easily be integrated into off-the-shelf ConvNet-based place recognition systems [12,32,14,33] for faster feature extraction. These methods build upon the use of holistic image descriptors, improving performance in different cases.…”

Section: G Integration Into Convnet-based Place Recognition Systemsmentioning

confidence: 99%

See 2 more Smart Citations

Lightweight Unsupervised Deep Loop Closure

Merrill¹,

Huang²

2018

Robotics: Science and Systems XIV

122

View full text Add to dashboard Cite

Abstract-Robust efficient loop closure detection is essential for large-scale real-time SLAM. In this paper, we propose a novel unsupervised deep neural network architecture of a feature embedding for visual loop closure that is both reliable and compact. Our model is built upon the autoencoder architecture, tailored specifically to the problem at hand. To train our network, we inflict random noise on our input data as the denoising autoencoder does, but, instead of applying random dropout, we warp images with randomized projective transformations to emulate natural viewpoint changes due to robot motion. Moreover, we utilize the geometric information and illumination invariance provided by histogram of oriented gradients (HOG), forcing the encoder to reconstruct a HOG descriptor instead of the original image. As a result, our trained model extracts features robust to extreme variations in appearance directly from raw images, without the need for labeled training data or environment-specific training. We perform extensive experiments on various challenging datasets, showing that the proposed deep loop-closure model consistently outperforms the state-of-the-art methods in terms of effectiveness and efficiency. Our model is fast and reliable enough to close loops in real time with no dimensionality reduction, and capable of replacing generic offthe-shelf networks in state-of-the-art ConvNet-based loop closure systems.

show abstract