HPILN: a feature learning framework for cross‐modality person re‐identification

Abstract

Most video surveillance systems use both RGB and infrared cameras, making it a vital technique to re‐identify a person cross the RGB and infrared modalities. This task can be challenging due to both the cross‐modality variations caused by heterogeneous images in RGB and infrared, and the intra‐modality variations caused by the heterogeneous human poses, camera position, light brightness etc. To meet these challenges, a novel feature learning framework, hard pentaplet and identity loss network (HPILN), is proposed. In the framework existing single‐modality re‐identification models are modified to fit for the cross‐modality scenario, following which specifically designed hard pentaplet loss and identity loss are used to increase the accuracy of the modified cross‐modality re‐identification models. Based on the benchmark of the SYSU‐MM01 dataset, extensive experiments have been conducted, showing that the authors’ method outperforms all existing ones in terms of cumulative match characteristic curve and mean average precision.

MSO: Multi-Feature Space Joint Optimization Network for RGB-Infrared Person Re-Identification

“…In this subsection, we compare our proposed method with several cross-modality person ReID methods that include the following categories: 1) With different structures and loss functions, Two-Stream, One-Stream, Zero-Padding [39], HSME, D-HSME [11], BDTR, SDL [19], DGD+MSR [7], EDFL [23], HPILN [50], AGW [44], cm-SSFT [25], and TSLFN+HC [54] learned modality-invariant feature representation; 2) With the ideas of GAN, cmGAN [4],…”

EXAM: A Framework of Learning Extreme and Moderate Embeddings for Person Re-ID

“…Compared with hand-crafted methods, deep learning approaches achieved a great improvement in recognition accuracy. However, these learned global representations mainly focuses on full body semantic and pays less attention to local details [ 8 ]. It naturally lacks flexible granularity for feature description and often suffers weak discriminative ability in identifying targets with similar inter-class common properties or large intra-class differences [ 9 ].…”

“…Deep neural network is originally developed for image classification [ 7 ], and its successful global feature learning strategy for classification was directly adopted for the person Re-ID approaches. The learned global representation pays less attention to local details [ 8 ], and often suffers weak discriminative ability in identifying targets with similar inter-class common properties or large intra-class differences [ 9 ]. For example, the following difficulties are encountered: (1) imprecise pedestrian detection affects global feature learning, e.g., shown in Figure 1 a; (2) body posture changes make the learning more difficult, e.g., Figure 1 b; (3) unexpected occlusion makes the learned features irrelevant to the human bodies, e.g., Figure 1 c; (4) cluttered background or multiple pedestrians with highly similar appearances make the model difficult to distinguish, e.g., Figure 1 d,e; (5) Misaligned bounding boxes make the model scale-variant, e.g., Figure 1 f.…”

MFCNet: Mining Features Context Network for RGB–IR Person Re-Identification

“…In the process of extracting features, skip connections are used to fuse the middle layers of the CNN model and enhance the robustness and non-descriptiveness of the extracted features. Zhao et al [27] expanded the triple loss function to pentaplet loss, and the cross-modality problem was considered on the basis of the original triple loss function; additionally, a method for mining difficult samples was introduced. Zhu et al [28] involves feature centers of the same category and the same modality, and hetero center loss was proposed on the basis of center loss, with a focus on the differences among feature centers of different modalities in the same category.…”