Gait-ViT: Gait Recognition with Vision Transformer

Abstract

Identifying an individual based on their physical/behavioral characteristics is known as biometric recognition. Gait is one of the most reliable biometrics due to its advantages, such as being perceivable at a long distance and difficult to replicate. The existing works mostly leverage Convolutional Neural Networks for gait recognition. The Convolutional Neural Networks perform well in image recognition tasks; however, they lack the attention mechanism to emphasize more on the significant regions of the image. The attention mechanism encodes information in the image patches, which facilitates the model to learn the substantial features in the specific regions. In light of this, this work employs the Vision Transformer (ViT) with an attention mechanism for gait recognition, referred to as Gait-ViT. In the proposed Gait-ViT, the gait energy image is first obtained by averaging the series of images over the gait cycle. The images are then split into patches and transformed into sequences by flattening and patch embedding. Position embedding, along with patch embedding, are applied on the sequence of patches to restore the positional information of the patches. Subsequently, the sequence of vectors is fed to the Transformer encoder to produce the final gait representation. As for the classification, the first element of the sequence is sent to the multi-layer perceptron to predict the class label. The proposed method obtained 99.93% on CASIA-B, 100% on OU-ISIR D and 99.51% on OU-LP, which exhibit the ability of the Vision Transformer model to outperform the state-of-the-art methods.

Ensemble of Heterogeneous Base Classifiers for Human Gait Recognition

“…However, the authors of those tests conducted them on a sample group of only 50 people which, as has been shown by [ 43 ], significantly impacts the results obtained. Equally high recognition rates have been presented in [ 36 ], where a method called Gait-ViT allowed the accurate recognition of 99.51 to 100 percent of gait cycles. In the work of [ 36 ], in contrast to [ 33 ], very good results were also gained for a database containing a larger number of people (OU-LP data set, 3916 participants).…”

“…Equally high recognition rates have been presented in [ 36 ], where a method called Gait-ViT allowed the accurate recognition of 99.51 to 100 percent of gait cycles. In the work of [ 36 ], in contrast to [ 33 ], very good results were also gained for a database containing a larger number of people (OU-LP data set, 3916 participants). It should be noted that the difficulty in the direct comparison of results lies in the fact that in both of the above-mentioned works, images from a video camera were used as data describing human gait.…”

“…The training time of this network for data contained within the CASIA-B database (124 people; less than 800 patterns) exceeded 9 min while the recognition time was 0.01 s. It is also worth mentioning that deep CNN is successfully used to classify various-sourced images as exemplified in the work of [ 35 ]. Article [ 36 ] presents the application of the Vision Transformer with an attention mechanism for gait recognition—or the GaitViT method. This instrument was tested on well-known databases, reaching 99.93% of correct classifications for the CASIA-B set, 100% for the OU-ISIR D, and 99.51% for the OU-LP databases.…”

Potential roles of transformers in brain tumor diagnosis and treatment

“…ViT successfully performs analyses by centralizing specific sites of the input image for better results [81] . Furthermore, investigations on ViT applications for the diagnosis and grading of various diseases have substantially increased.…”

“…[79,80] ViT successfully performs analyses by centralizing specific sites of the input image for better results. [81] Furthermore, investigations on ViT applications for the diagnosis and grading of various diseases have substantially increased. Researchers have established ViT-based architectures for diagnosing and evaluating corona virus disease 2019 (COVID-19) severity [82,83] and acute lymphoblastic leukemia [84] and grading diabetic retinopathy, [85] showing >90% classification accuracy.…”