Reinforced Temporal Attention and Split-Rate Transfer for Depth-Based Person Re-identification

Karianakis, Nikolaos; Liu, Zicheng; Chen, Yinpeng; Soatto, Stefano

doi:10.1007/978-3-030-01228-1_44

Cited by 39 publications

(29 citation statements)

References 106 publications

(118 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As to multi-modal methods, they usually combine skeleton-based features with extra RGB or depth information (e.g., depth shape features based on point clouds [38], [41], [42]) to boost Re-ID performance. In [43], CNN-LSTM with reinforced temporal attention (RTA) is proposed for person Re-ID based on a split-rate RGB-depth transfer approach.…”

Section: Person Re-identificationmentioning

confidence: 99%

Self-Supervised Gait Encoding with Locality-Aware Attention for Person Re-Identification

Rao

Wang

et al. 2020

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

Gait-based person re-identification (Re-ID) is valuable for safety-critical applications, and using only 3D skeleton data to extract discriminative gait features for person Re-ID is an emerging open topic. Existing methods either adopt hand-crafted features or learn gait features by traditional supervised learning paradigms. Unlike previous methods, we for the first time propose a generic gait encoding approach that can utilize unlabeled skeleton data to learn gait representations in a self-supervised manner. Specifically, we first propose to introduce self-supervision by learning to reconstruct input skeleton sequences in reverse order, which facilitates learning richer high-level semantics and better gait representations. Second, inspired by the fact that motion's continuity endows temporally adjacent skeletons with higher correlations (“locality”), we propose a locality-aware attention mechanism that encourages learning larger attention weights for temporally adjacent skeletons when reconstructing current skeleton, so as to learn locality when encoding gait. Finally, we propose Attention-based Gait Encodings (AGEs), which are built using context vectors learned by locality-aware attention, as final gait representations. AGEs are directly utilized to realize effective person Re-ID. Our approach typically improves existing skeleton-based methods by 10-20% Rank-1 accuracy, and it achieves comparable or even superior performance to multi-modal methods with extra RGB or depth information.

show abstract

Section: Person Re-identificationmentioning

confidence: 99%

Self-Supervised Gait Encoding with Locality-Aware Attention for Person Re-Identification

Rao

Wang

et al. 2020

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

show abstract

“…Wu et al [182] used the skeleton-based features to identify a person and, in particular, proposed an implicit feature transfer scheme when depth information was unavailable. Karianakis et al [183] and Hafner et al [184] mainly studied the cross-modal implementation from RGB-based image datasets to the RGB-D dataset because of their scale differences. There are also some several RGB-D datasets, such as RGBD-ID [157], BIWI RGBD-ID [158], and KinectREID [159], have been proposed to measure the performances of these studies.…”

Section: ) Others A: Depth-based Person Re-identificationmentioning

confidence: 99%

A Comprehensive Overview of Person Re-Identification Approaches

Wang

Zhao

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Person re-identification, identifying and tracking pedestrians in cross-domain monitoring systems, is an important technology in the computer vision field and of real significance for the construction of smart cities. With the development of deep learning techniques, especially convolutional neural networks, this technology has received more extensive attention and improvement in recent years and a large number of noteworthy achievements have emerged. This paper provides a comprehensive overview of person re-identification approaches to assist researchers in quickly understand this field with preference as well as to provide a more structured framework. By reviewing more than 300 re-identification related papers, the focus of these studies is summarized as information extraction, metric learning, post-processing, efficiency improvement, labeling cost reduction, and data type expansion. This classification is then organized based on different technologies, and on this basis, the pros and cons of each technology are analyzed. Moreover, this overview summarizes the difficulties and challenges of re-identification and discusses the possible research directions for reference. INDEX TERMSComputer vision, convolutional neural networks, distance learning, feature extraction, person re-identification.

show abstract

“…Person re-identification (Re-ID) aims to retrieve the same individual from a different view or scene, with great potential in authenticationrelated applications [28]. Conventional studies [10,14,29] typically extract appearance-based features such as body texture and silhouettes from RGB or depth images to perform person Re-ID. Nevertheless, an important flaw of these methods is their vulnerability to illumination or appearance changes [25].…”

Section: Introductionmentioning

confidence: 99%

SM-SGE: A Self-Supervised Multi-Scale Skeleton Graph Encoding Framework for Person Re-Identification

Rao¹,

Hu²,

Cheng³

et al. 2021

Preprint

View full text Add to dashboard Cite

Person re-identification via 3D skeletons is an emerging topic with great potential in security-critical applications. Existing methods typically learn body and motion features from the body-joint trajectory, whereas they lack a systematic way to model body structure and underlying relations of body components beyond the scale of body joints. In this paper, we for the first time propose a Self-supervised Multi-scale Skeleton Graph Encoding (SM-SGE) framework that comprehensively models human body, component relations, and skeleton dynamics from unlabeled skeleton graphs of various scales to learn an effective skeleton representation for person Re-ID. Specifically, we first devise multi-scale skeleton graphs with coarse-to-fine human body partitions, which enables us to model body structure and skeleton dynamics at multiple levels. Second, to mine inherent correlations between body components in skeletal motion, we propose a multi-scale graph relation network to learn structural relations between adjacent body-component nodes and collaborative relations among nodes of different scales, so as to capture more discriminative skeleton graph features. Last, we propose a novel multi-scale skeleton reconstruction mechanism to enable our framework to encode skeleton dynamics and high-level semantics from unlabeled skeleton graphs, which encourages learning a discriminative skeleton representation for person Re-ID. Extensive experiments show that SM-SGE outperforms most state-of-the-art skeleton-based methods. We further demonstrate its effectiveness on 3D skeleton data estimated from large-scale RGB videos. Our codes are open at https://github.com/Kali-Hac/SM-SGE.

show abstract

Reinforced Temporal Attention and Split-Rate Transfer for Depth-Based Person Re-identification

Cited by 39 publications

References 106 publications

Self-Supervised Gait Encoding with Locality-Aware Attention for Person Re-Identification

Self-Supervised Gait Encoding with Locality-Aware Attention for Person Re-Identification

A Comprehensive Overview of Person Re-Identification Approaches

SM-SGE: A Self-Supervised Multi-Scale Skeleton Graph Encoding Framework for Person Re-Identification

Contact Info

Product

Resources

About