Context-Sensitive Temporal Feature Learning for Gait Recognition

Abstract

Although gait recognition has drawn increasing research attention recently, it remains challenging to learn discriminative temporal representation, since the silhouette differences are quite subtle in spatial domain. Inspired by the observation that human can distinguish gaits of different subjects by adaptively focusing on temporal clips with different time scales, we propose a context-sensitive temporal feature learning (CSTL) network for gait recognition. CSTL produces temporal features in three scales, and adaptively aggregates them according to the contextual information from local and global perspectives. Specifically, CSTL contains an adaptive temporal aggregation module that subsequently performs local relation modeling and global relation modeling to fuse the multi-scale features. Besides, in order to remedy the spatial feature corruption caused by temporal operations, CSTL incorporates a salient spatial feature learning (SSFL) module to select groups of discriminative spatial features. Particularly, we utilize transformers to implement the global relation modeling and the SSFL module. To the best of our knowledge, this is the first work that adopts transformer in gait recognition. Extensive experiments conducted on three datasets demonstrate the state-of-the-art performance. Concretely, we achieve rank-1 accuracies of 98.7%, 96.2% and 88.7% under normal-walking, bag-carrying and coat-wearing conditions on CASIA-B, 97.5% on OU-MVLP and 50.6% on GREW.

GaitGL: Learning Discriminative Global-Local Feature Representations for Gait Recognition

“…The experimental results are shown in Table . 1. The compared methods include GaitSet [10], GaitPart [14], MT3D [13], 3D Local [45], CSTL [46] and our preliminary [19]. It can be observed that the proposed method achieves appealing performance compared with other SOTA methods.…”

“…Evaluation on OUMVLP. For the OUMVLP dataset, we compare the proposed method with several outstanding gait recognition methods, including GEINet [11], GaitSet [10], GaitPart [14], GLN [17], GaitKMM [49], SRN+CB [18], CSTL [46], 3D Local [45] and Our preliminary [19]. We take the same protocol as the GaitSet and GaitPart methods for fair comparison.…”

Gait Recognition in the Wild with Dense 3D Representations and A Benchmark

“…Then, gait patterns are modeled by parameters like lengths of limbs, angles of joints, and relative positions of body parts [3,48]. The model-free methods mainly adopt the silhouettes obtained by background subtraction from video frames [5,9,11,15,16,22,32,46,57,58]. In particular, Han et al proposed to aggregate a sequence of silhouettes into a compact Gait Energy Image (GEI) [11] which was widely used by the following methods [32,46].…”

“…For example, Shiraga et al [32] and Wu et al [46] proposed to learn effective features from GEIs and significantly outperformed previous methods. The most recent methods started to learn discriminative features directly from the silhouette sequences using larger CNNs or multi-scale structures and achieved state-of-the-art results [5,9,16,22]. Despite the excellent performance on in-the-lab datasets, e.g., CASIA-B and OU-LP, these methods usually fail in the wild as shown in the experiments on GREW [63] and our Gait3D.…”

“…5) GaitGL [22] is also a CNN-based framework to learn both global and local features from gait sequences. 6) CSTL [16] applies multi-scale learning on the temporal dimension of the sequence to learn both long-term and short-term motion for gait recognition.…”