Explicit feature disentanglement for visual place recognition across appearance changes

Abstract

In the long-term deployment of mobile robots, changing appearance brings challenges for localization. When a robot travels to the same place or restarts from an existing map, global localization is needed, where place recognition provides coarse position information. For visual sensors, changing appearances such as the transition from day to night and seasonal variation can reduce the performance of a visual place recognition system. To address this problem, we propose to learn domain-unrelated features across extreme changing appearance, where a domain denotes a specific appearance condition, such as a season or a kind of weather. We use an adversarial network with two discriminators to disentangle domain-related features and domain-unrelated features from images, and the domain-unrelated features are used as descriptors in place recognition. Provided images from different domains, our network is trained in a self-supervised manner which does not require correspondences between these domains. Besides, our feature extractors are shared among all domains, making it possible to contain more appearance without increasing model complexity. Qualitative and quantitative results on two toy cases are presented to show that our network can disentangle domain-related and domain-unrelated features from given data. Experiments on three public datasets and one proposed dataset for visual place recognition are conducted to illustrate the performance of our method compared with several typical algorithms. Besides, an ablation study is designed to validate the effectiveness of the introduced discriminators in our network. Additionally, we use a four-domain dataset to verify that the network can extend to multiple domains with one model while achieving similar performance.

A systematic literature review on long‐term localization and mapping for mobile robots

“…This results indicates that viewpoint change is considered to be content in the proposed algorithm. With a similar architecture to C. Qin et al (2020), Tang et al (2021) also consider a place domain discriminator to ensure that the content discriminator only contains the place information and does not include the location appearance. The method uses data augmentation in training to increase the robustness of the content discriminator against viewpoint changes.…”

Self-Supervised Learning for Place Representation Generalization across Appearance Changes

“…Despite this, only a few methods have been developed to this day. For instance, Tang et al [42] have proposed to disentangle appearance-related and place-related features using a generative adversarial network with two discriminators. However, this type of method may suffer from unstable training.…”

“…In practice, they are designed to obtain image representations that are sensitive and/or robust to given image transformations without requiring any type of manual annotation. While only a few works have investigated SSL for VPR [12,42], we herein propose to combine the two main SSL paradigms, i.e., Contrastive Learning (CL) [6] and Predictive Learning (PL) [20], to obtain image representations that are both robust to appearance changes and sensitive to geometric transformations. By doing that, our goal is to learn features suitable for visual place recognition under appearance changes.…”

A Systematic Literature Review on Long-Term Localization and Mapping for Mobile Robots

“…This results indicated that viewpoint change is considered to be content in the proposed algorithm. With a similar architecture to C. Qin et al 2020, Tang, Y. Wang, Tan, et al 2021 also considers a place domain discriminator to ensure that the content discriminator only contains the place information and not also its appearance, while also using data augmentation in training to increase robustness against viewpoint changes. In the experiments, all images generated from a zero-appearance feature vector looked similar, while their place information remains conserved indicating that the proposed method can disentangle the input image across appearance changes.…”

“…One possible direction could be the usage of data augmentation as in Tang, Y. Wang, Tan, et al 2021 for learning global visual descriptors, even though the latter work does not clarify to what extent augmented data helped in viewpoint variance. Another possible solution would be the use of omnidirectional vision, even though the networks traditionally used for learning CNN-based features (considered more discriminative compared to handcrafted features, as previously discussed in Section 5.1.4) may not be directly applicable due to the different aspect ratio of omnidirectional images retrieve from sensors such as the Point Grey LadyBug 2 5-view.…”