Modality Discrepancy Reduction for Visible-Infrared Person Re-Identification under Complete Modality Missing

Abstract

Person re-identification (Re-ID) aims to match person images across non-overlapping camera views. Most of existing methods focus on short-time and small-scale surveillance systems in which each person is captured in multiple camera views of adjacent scenes in a short time span. However, in long-term and large-scale surveillance systems covering larger areas and spanning different time duration, existing methods tend to fail. Specifically, when light condition is extremely dark, the camera automatically switches to infrared mode. Matching across infrared images and RGB images is difficult due to the large appearance difference. Meanwhile, since most pedestrians appear in limited local areas and fixed time, it is difficult to collect training images of the same person under both infrared and RGB mode. Hence, a model is required to match person images cross modality under a complete modality missing condition. In this work, we study intra-modality supervised person re-identification under complete modality missing, which uses cross-modality unpaired data with intra-modality identity labels for training. It is challenging as cross-modality paired data plays an important role for learning modality-invariant representation in most existing Re-ID methods. To learn modality-invariant representation from cross-modality unpaired training data, we first introduce a strong baseline with a dual-head cross-entropy loss and a multi-modality negative loss, aiming to alleviate cross-modality contrast and enhance intra-modality contrast. Then, we propose a residual modality alleviation network and a shape-guided consistency learning loss to further alleviate cross-modality representation discrepancy. The experiments are conducted in the complete modality missing setting on SYSY-MM01 and RegDB datasets. The evaluation results demonstrate the superiority of our method.