Visible-Infrared Person Re-Identification: A Comprehensive Survey and a New Setting

Abstract

Person re-identification (ReID) plays a crucial role in video surveillance with the aim to search a specific person across disjoint cameras, and it has progressed notably in recent years. However, visible cameras may not be able to record enough information about the pedestrian’s appearance under the condition of low illumination. On the contrary, thermal infrared images can significantly mitigate this issue. To this end, combining visible images with infrared images is a natural trend, and are considerably heterogeneous modalities. Some attempts have recently been contributed to visible-infrared person re-identification (VI-ReID). This paper provides a complete overview of current VI-ReID approaches that employ deep learning algorithms. To align with the practical application scenarios, we first propose a new testing setting and systematically evaluate state-of-the-art methods based on our new setting. Then, we compare ReID with VI-ReID in three aspects, including data composition, challenges, and performance. According to the summary of previous work, we classify the existing methods into two categories. Additionally, we elaborate on frequently used datasets and metrics for performance evaluation. We give insights on the historical development and conclude the limitations of off-the-shelf methods. We finally discuss the future directions of VI-ReID that the community should further address.

A person re‐identification method for sports event scenes incorporating textual information mining

“…In light of the challenges posed by low-light conditions and their impact on the adequate capture of comprehensive person appearance information using a single visible light camera, the integration of visible light and infrared images has emerged as a trend in person re-identification research. Zheng et al [22] conducted a comprehensive and meticulous study, offering an in-depth survey of prevailing methodologies concerning the fusion of visible and infrared light data. They embarked on a detailed examination of various facets related to image fusion, encompassing crucial aspects such as data structure, encountered challenges, and performance evaluation metrics.…”

Person Re-Identification with RGB–D and RGB–IR Sensors: A Comprehensive Survey

“…Single-modal person Re-ID matches probe samples with gallery samples, and all samples are taken from the same modality (i.e., RGB–RGB or IR–IR matching) [ 65 ]. Unlike single-modal Re-ID, cross-modal Re-ID aims to match the probe sample taken from one modality against a gallery set from another modality, such as RGB–IR, RGB–depth, or sketch–RGB images [ 13 , 52 , 66 ], as shown in Figure 10 .…”

“…RGB–IR-based Person Re-ID is the most widely studied cross-modal setting over all the other alternatives, thanks to the introduction of the SYSU-MM01 dataset [ 45 ], which initiates the path of an RGB–IR-based cross-modal Re-ID scenario. Following the survey paper in [ 13 ], state-of-the-art Re-ID approaches using RGB–IR-based cross-modal methods can be divided into two categories: non-generative- [ 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 , 80 , 81 ] and generative-based approaches. The former one relies on traditional feature representation [ 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 , 80 , 81 ] and metric learning approaches to maximize the similarities between two images with the same identity and minimize the similarities between two images with different identities, while the latter one depends on the unification of images from different modalities to minimize the data distribution gap between two different modalities.…”

“…However, these surveys mainly focus on and give insightful direction on RGB camera-based person Re-ID approaches. Recently, Zheng et al [ 13 ] published a survey that focused only on RGB–IR sensor-based Re-ID methods.…”

Learning Global Average Attention Pooling (GAAP) on Resnet50 Backbone for Person Re-identification Problem

“…Similar Works considered body reconstruction models and distance metric learning [8]. In the last decade, the growth of AI-based approaches has captured the Pe-reID problem and has subsequently proved its potential with exceptionally good recognition accuracies across datasets [9]. However, the results obtained were nowhere close to the requirements of a real-time deployment pipeline.…”