Feature Fusion and Center Aggregation for Visible-Infrared Person Re-Identification

Abstract

The visible-infrared pedestrian re-identification (VI Re-ID) task aims at matching crossmodality pedestrian images with the same labels. Most current methods focus on mitigating the modality discrepancy by adopting a two-stream network and identity supervision. Based on current methods, we propose a novel feature fusion and center aggregation learning network (F 2 CALNet) for cross-modality pedestrian re-identification. F 2 CALNet focuses on learning modality-irrelevant features by simultaneously reducing inter-modality discrepancies and increasing the inter-identity variations in a single framework. Specifically, we first adopt a two-stream backbone network to extract modality-independent information and modality-shared information. Then, we embed modality mitigation modules in a two-stream network to learn feature maps that are stripped of the modality information. Finally, we devise a feature fusion and center aggregation learning module, which first merges two different granularity features to learn distinguishing features, then, we organize two kinds of center-based loss functions to reduce the intraidentity inter-and intra-modality differences and increase inter-identity variations by simultaneously pulling the features of the same identity close to their centers and pushing far away the centers of different identities. Extensive experiments on two public cross-modality datasets (SYSU-MM01 and RegDB) show that F 2 CALNet is superior to the state-of-the-art approaches. Furthermore, on the SYSU-MM01 datasets, our model outperforms the baseline by 5.52% and 4.25% for the accuracy of rank1 and mAP, respectively.

Global and Part Feature Fusion for Cross-Modality Person Re-Identification

“…Based on the results of [7], we empirically set the dual-stream network as our backbone to process images across modalities, as shown in Figure 2. Following [26], [27], the ResNet50 model [28] is adopted to construct the backbone network. The three remaining res-convolution modules (blocks2, 3 and 4) are configured as modal-sharing sub-modules with shared parameters to learn multi-modal shared middle layer feature representations in a shared 3D feature space [8].…”