Gait Recognition with Multiple-Temporal-Scale 3D Convolutional Neural Network

Abstract

Gait recognition which is one of the most important and effective biometric technologies has a significant advantage in long-distance recognition systems. For existing gait recognition methods, the template-based approaches may lose temporal information, while the sequence-based methods cannot fully exploit the temporal relations among the sequence. To address the above issues, we propose a novel multiple-temporal-scale gait recognition framework which integrates the temporal information in multiple temporal scales, making use of both the frame and interval fusion information. Moreover, the interval-level representation is realized by a local transformation module. Concretely, 3D convolution neural network (3D CNN) is applied in both the small and the large temporal scales to extract the spatial-temporal information. Moreover, a frame pooling method is developed to address the mismatch of the input of 3D network and video frames, and a novel 3D basic network block is designed to improve efficiency. Experiments demonstrate that the multiple-temporal-scale 3D CNN based gait recognition method can achieve better performance than most recent state-of-the-art methods in CASIA-B dataset. The proposed method obtains the rank-1 accuracy with 96.7% under normal condition, and outperforms other methods on average accuracy by at least 5.8% and 11.1%, respectively, in complex scenarios. CCS CONCEPTS • Security and privacy → Biometrics.

Silhouette-Based View-Embeddings for Gait Recognition Under Multiple Views

“…Fan et al [12] further improved silhouette gait feature by introducing part-based representations. Lin et al [14,17] proposed a comprehensive model named GaitGL by integrating both global and local feature. It is recognized as the state-of-the-art gait recognition method.…”

“…Pipeline of our method. The backbone can be replaced by any silhouette-based network, such as Gaitset [11], Gaitpart [12], MT3D [14] and GaitGL [17]. For the extracted feature map, we feed it into two branches.…”

“…Gait recognition methods include model-based [1,2] and appearance-based methods. Appearance-based methods can be divided into two categories, i.e., regarding gait as a single image [3,4,5,6,7,8,9,10] and regarding gait as a video or an image sequence [11,12,13,14,15]. The first category often use gait energy images (GEIs) [16] as the representation of gait, while the second category directly makes use of the gait silhouette images.…”

Residual connection-based graph convolutional neural networks for gait recognition

“…A combination of hand-crafted geometric features and artificial neural network has proven to be effective for Kinect-based gait recognition [25]. A multitemporal 3D Convolution Neural Network (3D-CNN) and a frame pooling method were recently proposed to handle gait recognition from videos [26]. Another work designed 3D-CNN-based method with spatial and temporal fusion for action recognition [27].…”

Multi-Scale Temporal Information Extractor For Gait Recognition

“…Considering that 3D convolution runs slower than 2D convolution, in this paper, We first use 2D convolution to extract frame-level features rapidly, and then use 3D convolution to extract both spatial and temporal information for more comprehensive representation. Moreover, to address the inflexibility of 3D convolution, we introduce an operation, called temporal aggregation [12,13], to integrate the whole temporal information from unfixed-length gait feature maps. Specifically, to better leverage spatial information, we first partition the feature map into multiple horizontal columns and then aggregate each column.…”