Semantically Self-Aligned Network for Text-to-Image Part-aware Person Re-identification

Abstract

Text-to-image person re-identification (ReID) aims to search for images containing a person of interest using textual descriptions. However, due to the significant modality gap and the large intra-class variance in textual descriptions, text-to-image ReID remains a challenging problem. Accordingly, in this paper, we propose a Semantically Self-Aligned Network (SSAN) to handle the above problems. First, we propose a novel method that automatically extracts semantically aligned part-level features from the two modalities. Second, we design a multi-view nonlocal network that captures the relationships between body parts, thereby establishing better correspondences between body parts and noun phrases. Third, we introduce a Compound Ranking (CR) loss that makes use of textual descriptions for other images of the same identity to provide extra supervision, thereby effectively reducing the intra-class variance in textual features. Finally, to expedite future research in text-to-image ReID, we build a new database named ICFG-PEDES. Extensive experiments demonstrate that SSAN outperforms state-of-the-art approaches by significant margins. Both the new ICFG-PEDES database and the SSAN code are available at https://github. com/zifyloo/SSAN .

Learning Granularity-Unified Representations for Text-to-Image Person Re-identification

“…LGUR achieves Rank-1 improvements of 9.53% and 8.76% on the C→I and I→C settings respectively when compared to SSAN [6].…”

“…Comparison results are summarized in Table 2. Since ICFG-PEDES is a new database, we directly cite the performance of existing approaches evaluated in [6].…”

“…Specifically, it achieves 57.42% and 59.02% Rank-1 accuracies with the ResNet-50 and DeiT-Small backbones, respectively. SSAN [6] achieves superior performance since it extracts fine-grained part-level textual and visual features. However, this method still ignores the granularity gap between the two modalities.…”

“…As discussed in Section 3.3, our LGUR has advantages in time complexity. In this subsection, we evaluate the training time 1 , inference time 2 and Rank-1 accuracy of LGUR, three works that do not implement the cross-modal attention mechanism (i.e., Dual Path [47], CMPM/C [45], and SSAN [6]), and another three methods that adopt cross-attention mechanism (i.e., MIA [25], SCAN [18], and NAFS [9]). To facilitate fair comparison, we set the image size of input images to 384 × 128 pixels and the batch size to 64 for all methods.…”

“…We conduct extensive experiments on two existing large-scale benchmark datasets, i.e., CUHK-PEDES [22] and ICFG-PEDES [6]. The results show that our simple LGUR framework consistently and significantly outperforms existing approaches.…”