Self-Supervised Pre-Training for Transformer-Based Person Re-Identification

Abstract

Transformer-based supervised pre-training achieves great performance in person re-identification (ReID). However, due to the domain gap between ImageNet and ReID datasets, it usually needs a larger pre-training dataset (e.g. ImageNet-21K) to boost the performance because of the strong data fitting ability of the transformer. To address this challenge, this work targets to mitigate the gap between the pre-training and ReID datasets from the perspective of data and model structure, respectively. We first investigate self-supervised learning (SSL) methods with Vision Transformer (ViT) pretrained on unlabelled person images (the LUPerson dataset), and empirically find it significantly surpasses ImageNet supervised pre-training models on ReID tasks. To further reduce the domain gap and accelerate the pre-training, the Catastrophic Forgetting Score (CFS) is proposed to evaluate the gap between pre-training and fine-tuning data. Based on CFS, a subset is selected via sampling relevant data close to the down-stream ReID data and filtering irrelevant data from the pre-training dataset. For the model structure, a ReID-specific module named IBN-based convolution stem (ICS) is proposed to bridge the domain gap by learning more invariant features. Extensive experiments have been conducted to fine-tune the pre-training models under supervised learning, unsupervised domain adaptation (UDA), and unsupervised learning (USL) settings. We successfully downscale the LUPerson dataset to 50% with no performance degradation. Finally, we achieve state-of-the-art performance on Market-1501 and MSMT17. For example, our ViT-S/16 achieves 91.3%/89.9%/89.6% mAP accuracy on Market1501 for supervised/UDA/USL ReID. Codes and models will be released to https://github.com/michuanhaohao/ TransReID-SSL.

DC-Former: Diverse and Compact Transformer for Person Re-Identification

“…TransReID (He et al 2021) is the first pure transformerbased method on re-ID, it proposes a jigsaw patches module (JPM) which shuffles patch embeddings and re-groups them for further feature learning to extract several local features and aggregates them to get robust feature with global context. TransReID-SSL (Luo et al 2021) uses a massive person re-ID dataset LU P erson (Fu et al 2021) to train a stronger pre-trained model by DINO (Caron et al 2021).…”

Meta Distribution Alignment for Generalizable Person Re-Identification

“…Based on ViT [41], [39] applies pure Transformer to supervised ReID for the first time, which introduces side information to improve the robustness of features. [47] further proposed self-supervised pre-training for Transformer-based person ReID, which mitigates the gap between the pre-training and ReID datasets from the perspective of data and model structure.…”

“…Vision Transformer usually yields better generalization ability than common CNN networks under distribution shift [49]. However, existing pure transformerbased ReID models are only used in supervised and pretrained ReID [47,39]. The generalization of Transformer is still unknown in DG ReID.…”

SoccerNet 2022 Challenges Results

“…We utilize non-parametric InfoNCE loss within batches. We use ViT with ICS in TransReID-SSL [22] as our backbone. Based on self-attention in Transformer, a features fusion method is proposed to generate the refined feature for an original query feature.…”