VRSTC: Occlusion-Free Video Person Re-Identification

Abstract

Video person re-identification (re-ID) plays an important role in surveillance video analysis. However, the performance of video re-ID degenerates severely under partial occlusion. In this paper, we propose a novel network, called Spatio-Temporal Completion network (STCnet), to explicitly handle partial occlusion problem. Different from most previous works that discard the occluded frames, STCnet can recover the appearance of the occluded parts. For one thing, the spatial structure of a pedestrian frame can be used to predict the occluded body parts from the unoccluded body parts of this frame. For another, the temporal patterns of pedestrian sequence provide important clues to generate the contents of occluded parts. With the spatio-temporal information, STCnet can recover the appearance for the occluded parts, which can be leveraged with those unoccluded parts for more accurate video re-ID. By combining a re-ID network with STCnet, a video re-ID framework robust to partial occlusion (VRSTC) is proposed. Experiments on three challenging video re-ID databases demonstrate that the proposed approach outperforms the state-of-the-art approaches.

Occlusion Detection for Automatic Video Editing

“…They divided the pedestrian region into five areas based on the empirical ratio [7], and estimated the occlusion status of each area using the micro neural network equipped in the proposed unit. Hou et al [13] believed that occlusion usually occurred in successive frames, and the occluders have different semantic features from the original body parts. So Hou et al calculated the cosine similarity between frame region feature and video region feature as a criteria score, and the area is considered as being occluded when the score is lower than the prescribed threshold.…”

“…Before feature extraction, we resize each video frame to 128 × 171 pixels. Limited to the 4GB RAM of NVIDIA GTX 1050Ti GPU, the [29] 139.59M 63.3% DenseNet-121 [15] 6.95M 64.9% Zhang et al [45] 0.04M 44.5% Hou et al [13] 23.51M 38.0% R(2+1)D [33] 33.18M 57.4% C3D(single output) [32] 107.36M 74.8% C3D(multiple output) [32] 107.36M 77.7% Our Method(without correction) 59.64M 82.1% Our Method(with correction) 59.64M 84.0%…”

“…If the visibility score of any region is less than the 0.65 set in the original paper, we consider that the frame is occluded. For the task of video person re-identification, Hou et al [13] used the similarity score to determine whether the occlusion occurred. According to their method, each frame are divided into three regions (upper, middle, and lower) and the ResNet-50 [11] trained on our occlusion detection dataset was utilized to extract the features of each region, called region feature.…”

Uncertainty-optimized deep learning model for small-scale person re-identification

“…For small-scale person re-identification, there are relatively few deep learning methods, most of which focus on how to expand the training data [36][37][38][39]. Chen et al [40] proposed a cross-domain architecture that could use an auxiliary training set.…”

Bodyprint—A Meta-Feature Based LSTM Hashing Model for Person Re-Identification

“…This leads to a first class of problems since, as known, visual features have many weaknesses, including illumination changes, shadows, direction of light, and many others. Another class of problems regards the background clutter [ 28 , 29 ] and occlusions [ 30 , 31 ], which tend, in uncontrolled environments, to lower system performances in term of accuracy. A final class of problems, very important from a practical point of view, is referred to the long-term re-identification and camouflage [ 32 , 33 ].…”