Gait Recognition in the Wild with Multi-hop Temporal Switch

Abstract

Existing studies for gait recognition are dominated by in-the-lab scenarios. Since people live in real-world senses, gait recognition in the wild is a more practical problem that has recently attracted the attention of the community of multimedia and computer vision. Current methods that obtain state-of-the-art performance on inthe-lab benchmarks achieve much worse accuracy on the recently proposed in-the-wild datasets because these methods can hardly model the varied temporal dynamics of gait sequences in unconstrained scenes. Therefore, this paper presents a novel multi-hop temporal switch method to achieve effective temporal modeling of gait patterns in real-world scenes. Concretely, we design a novel gait recognition network, named Multi-hop Temporal Switch Network (MTSGait), to learn spatial features and multi-scale temporal features simultaneously. Different from existing methods that use 3D convolutions for temporal modeling, our MTSGait models the temporal dynamics of gait sequences by 2D convolutions. By this means, it achieves high efficiency with fewer model parameters and reduces the difficulty in optimization compared with 3D convolution-based models. Based on the specific design of the 2D convolution kernels, our method can eliminate the misalignment of features among adjacent frames. In addition, a new sampling strategy, i.e., non-cyclic continuous sampling, is proposed to make the model learn more robust temporal features. Finally, the proposed method achieves * This work was done when Jinkai Zheng was an intern at JD Explore Academy.

HybridGait: A Benchmark for Spatial-Temporal Cloth-Changing Gait Recognition with Hybrid Explorations

“…In the model-based method category, we analyze GaitGraph (Teepe et al 2021), which utilizes a 2D skeleton as a graph and inputs it into a Graph Convolution Network. In the appearance-based method category, we examine GEINet (Shiraga et al 2016), GaitSet (Chao et al 2019), GaitGL (Lin, Zhang, and Yu 2021), GaitPart (Fan et al 2020), CSTL (Huang et al 2021a), MTSGait (Zheng et al 2022a) and GaitGCI (Dou et al 2023). These models use Convolutional Neural Networks (CNNs) to learn features from various sources, including GEIs, silhouettes, and gait sequences.…”

Full-body pose reconstruction and correction in virtual reality for rehabilitation training

“…Inspired by previous study (Zheng et al, 2022 ), we conduct K forward feature exchange modules alone the temporal direction for each feature slice F t . In the branch where K = k , we exchange the features and in the feature slice F t with the corresponding features and in the feature slice F t + k , where F t + k represents the feature slice at a temporal distance of k frames, as shown in Equation 8 :…”

Effective Fusion Method on Silhouette and Pose for Gait Recognition

“…To enhance the temporal representation ability, the temporal representation, a multi-scale model [29] is proposed where the small-temporal-scale branch is used to model slow changes, while the larger-temporal-scale one is designed to grasp the rapid gait changes. Recent development seen a lot new ideas emerging, for instance, instead of extracting the spatial-temporal information, Gaithop [30] focuses on the channel information by switching channel of different frames. In addition, an end-to-end model is proposed by GaitEdge [31] which combines the silhouette extraction and the gait recognition model into one pipeline and achieves significant improvement over the separate pipeline.…”