From Face to Gait: Weakly-Supervised Learning of Gender Information from Walking Patterns

Abstract

Gaze estimation, the task of predicting where an individual is looking, is a critical task with direct applications in areas such as human-computer interaction and virtual reality. Estimating the direction of looking in unconstrained environments is difficult, due to the many factors that can obscure the face and eye regions. In this work we propose CrossGaze, a strong baseline for gaze estimation, that leverages recent developments in computer vision architectures and attention-based modules. Unlike previous approaches, our method does not require a specialized architecture, utilizing already established models that we integrate in our architecture and adapt for the task of 3D gaze estimation. This approach allows for seamless updates to the architecture as any module can be replaced with more powerful feature extractors. On the Gaze360 benchmark, our model surpasses several state-of-the-art methods, achieving a mean angular error of 9.94 • . Our proposed model serves as a strong foundation for future research and development in gaze estimation, paving the way for practical and accurate gaze prediction in real-world scenarios.

Learning Gait Representations with Noisy Multi-Task Learning

“…Background. We explore spatial-temporal graph classification in the context of gait-based gender estimation [4] using sequences of skeletons extracted by a pretrained pose estimator network [13]. Given a dataset D = {( ŷi , J i )} finding the best model f θ for the classification task corresponds to finding the optimal parameters θ which minimize the cross-entropy loss L across the dataset: θ…”

“…This noninvasive method of biometric processing has found significant applications across several domains, including security and surveillance [9] and healthcare [12]. Gait patterns offer insights into the identity of a person [9], demographics [4], emotions [24] and mental state [10].…”

“…Our model is trainable end-to-end by using the Straight-Through Gumbel-Softmax trick [17]. We conduct our experiments on CASIA-B for the task of gait-based gender estimation [4], as this is a sufficiently simple task to demonstrate the effectiveness of our approach. Importantly, we show that the resulting graphs are more interpretable and qualitatively different from fixed graphs used by ST-GCN [29] and MS-G3D [21].…”

“…Cosma et al [11] used an ST-GCN to learn discriminative gait features in a self-supervised manner. Catruna et al [5] used a hierarchical transformer that utilizes the anatomical structure of the human body to construct discriminative gait features. However, these methods operate on a fixed, hand-crafted spatio-temporal graph which might be sub-optimal across all tasks and datasets.…”

Integration of Face and Gait Recognition via Transfer Learning: A Multiscale Biometric Identification Approach

“…Recent years have seen the emergence of methods integrating face and gait recognition, combining physical and behavioural biometrics to explore whether this amalgamation can enhance the performance of systems that utilise only one of these biometrics [15,16]. Although this multi-biometric combination is still in its nascent stage, with relatively few studies published on the topic, the findings thus far are promising, showing clear potential for this approach in differentiating individuals [17].…”