Visual Sequence Place Recognition with Improved Dynamic Time Warping

Lu, F.; Chen, Baifan; Guo, Zhaohong; Zhou, Xiangdong

doi:10.1109/ictai.2019.00145

Cited by 6 publications

(7 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Now we will introduce the image sequence distance measurement based on the idea of temporal alignment. The basic algorithm (LM-DTW) is first proposed in our previous work [15], in which we call it "improved DTW". As can be appreciated in Fig.…”

Section: Temporal Alignment For Image Sequence Matchingmentioning

confidence: 99%

“…5. The details are as follows: The Nordland and Gardens Point (GP) Datasets are used in our previous work [15]. But this time the reference and query datasets we used have the same number of images.…”

Section: A Datasets Performance Evaluation and Parameter Setupmentioning

confidence: 99%

“…Follow this line, we also align the images from the temporal domain, i.e., utilize image sequence for VPR. The local matching DTW (LM-DTW) algorithm is used to perform image sequence matching, which is actually proposed in our previous work [15]. Since most sequence-based place recognition methods use global descriptors to represent images, which are condition-invariant but not viewpoint-invariant.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

STA-VPR: Spatio-temporal Alignment for Visual Place Recognition

Lu,

Chen,

Zhou

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Recently, the methods based on Convolutional Neural Networks (CNNs) have gained popularity in the field of visual place recognition (VPR). In particular, the features from the middle layers of CNNs are more robust to drastic appearance changes than handcrafted features and high-layer features. Unfortunately, the holistic mid-layer features lack robustness to large viewpoint changes. Here we split the holistic mid-layer features into local features, and propose an adaptive dynamic time warping (DTW) algorithm to align local features from the spatial domain while measuring the distance between two images. This realizes viewpoint-invariant and conditioninvariant place recognition. Meanwhile, a local matching DTW (LM-DTW) algorithm is applied to perform image sequence matching based on temporal alignment, which achieves further improvements and ensures linear time complexity. We perform extensive experiments on five representative VPR datasets. The results show that the proposed method significantly improves the CNN-based methods. Moreover, our method outperforms several state-of-the-art methods while maintaining good runtime performance. This work provides a novel way to boost the performance of CNN methods without any re-training for VPR. The code is available at https://github.com/Lu-Feng/STA-VPR.

show abstract

Section: Temporal Alignment For Image Sequence Matchingmentioning

confidence: 99%

Section: A Datasets Performance Evaluation and Parameter Setupmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

STA-VPR: Spatio-temporal Alignment for Visual Place Recognition

Lu,

Chen,

Zhou

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Their results showed significant performance improvement as compared to the individual performance of the mentioned descriptors and the ABLE methods. Later, Lu et al have presented in [14] a sequence place recognition using an improved DTW.…”

Section: Introductionmentioning

confidence: 99%

Dynamic Time Warping of Deep Features for Place Recognition in Visually Varying Conditions

2021

View full text Add to dashboard Cite

This paper presents a new visual place recognition (VPR) method based on dynamic time warping (DTW) and deep convolutional neural network. The proposal considers visual place recognition in environments that exhibit changes in several visual conditions like appearance and viewpoint changes. The proposed VPR method belongs to the sequence matching category, i.e., it utilizes the sequence-to-sequence image matching to recognize the best matching to the current test image. This approach extracts the image's features from a deep CNN, where different layers of a two selected CNNs are investigated and the best performing layer along with the DTW is identified. Also, the performance of the deep features is compared to the one of classical features (handcrafted features like SIFT, HOG and LDB). Our experiments also compare the performance with other state-of-the-art visual place recognition algorithms, Holistic, Only look once, NetVLAD and SeqSLAM in particular.

show abstract

“…With the development of deep learning, the methods based on deep convolutional features outperform traditional handcraft features in many tasks in the field of computer vision. Features extracted through convolutional neural network (CNN) are deeper and more abstract, thus being non-sensitive to environmental conditions and appearance variations [15][16][17]. Chen et al [18] applied the image features extracted by CNN to VPR, verifying the effectiveness of convolutional neural network in place recognition.…”

Section: Introductionmentioning

confidence: 99%

Hierarchical Visual Place Recognition Based on Semantic-Aggregation

et al. 2021

Self Cite

View full text Add to dashboard Cite

A major challenge in place recognition is to be robust against viewpoint changes and appearance changes caused by self and environmental variations. Humans achieve this by recognizing objects and their relationships in the scene under different conditions. Inspired by this, we propose a hierarchical visual place recognition pipeline based on semantic-aggregation and scene understanding for the images. The pipeline contains coarse matching and fine matching. Semantic-aggregation happens in residual aggregation of visual information and semantic information in coarse matching, and semantic association of semantic edges in fine matching. Through the above two processes, we realized a robust coarse-to-fine pipeline of visual place recognition across viewpoint and condition variations. Experimental results on the benchmark datasets show that our method performs better than several state-of-the-art methods, improving the robustness against severe viewpoint changes and appearance changes while maintaining good matching-time performance. Moreover, we prove that it is possible for a computer to realize place recognition based on scene understanding.

show abstract

Visual Sequence Place Recognition with Improved Dynamic Time Warping

Cited by 6 publications

References 28 publications

STA-VPR: Spatio-temporal Alignment for Visual Place Recognition

STA-VPR: Spatio-temporal Alignment for Visual Place Recognition

Dynamic Time Warping of Deep Features for Place Recognition in Visually Varying Conditions

Hierarchical Visual Place Recognition Based on Semantic-Aggregation

Contact Info

Product

Resources

About