Spatiotemporal Analysis by Deep Learning of Gait Signatures From Floor Sensors

Abstract

The recognition of gait pattern variation is of high importance to various industrial and commercial applications, including security, sport, virtual reality, gaming, robotics, medical rehabilitation, mental illness diagnosis, space exploration, and others. The purpose of this paper is to study the nature of gait variability in more detail, by identifying gait intervals responsible for gait pattern variations in individuals, as well as between individuals, using cognitive demanding tasks. This work uses deep learning methods for sensor fusion of 116 plastic optical fiber (POF) distributed sensors for gait recognition. The floor sensor system captures spatiotemporal samples due to varying ground reaction force (GRF) in multiples of up to 4 uninterrupted steps on a continuous 2x1 m area. We demonstrate classifications of gait signatures, achieving up to 100% F1-score with Convolutional Neural Networks (CNN), in the context of gait recognition of 21 subjects, with imposters and clients. Classifications under cognitive load, induced by 4 different dual tasks, manifested lower F1scores. Layer-Wise Relevance Propagation (LRP) methods are employed to decompose a trained neural network prediction to relevant standard events in the gait cycle, by generating a "heat map" over the input used for classification. This allows valuable insight into which parts of the gait spatiotemporal signal have the heaviest influence on the gait classification and consequently, which gait events, such as heel strike or toe-off, are mostly affected by cognitive load.

Human gait analysis based on automatic recognition: A review

“…Over the last few years, recent work floor sensors have been applied for medical applications such as the impact of muscle fatigue on gait characteristics, health monitoring, and age-based classification (Alharthi et al, 2021); as well as characterization of gait abnormalities in multiple sclerosis, Parkinsons disease, or fibromyalgia patients (Klöpfer-Krämer et al, 2020). A force platform (or a force plate), integrates devices with either strain gauges or piezoelectric transducers.…”

“…However, since many individuals may have the same body mass, the recognition rate may below. The floor sensor system captures spatiotemporal samples due to varying ground reaction force (GRF) in multiples of up to 4 uninterrupted steps on a continuous area (Alharthi et al, 2021).…”

Emerging Trends in Sports and Artificial Intelligence: A Scientometric Analysis in Citespace

“…Additionally, wearable devices are a commonly used technology in this field for data collection about health and exercise. Most of this data is further processed using a machine learning algorithm [19,20]. CiteSpace is a tool that presents the literature as a network of multiple interconnected sub-networks, called 'time slices,' each built using articles published within a year.…”

“…Additionally, wearable devices are a commonly used technology in this field for data collection about health and exercise. Most of this data is further processed using a machine learning algorithm[19,20].…”

Pattern recognition of gait signals using φ-OTDR

“…A deep neural network can be trained to recognize the most significant changes in a person's gait pattern from a small group of flagged multi-angle videos of pedestrians walking, and thus identify a person. Alharthi et al [11] developed a biometric verification system to measure human gait and walking patterns. As long as an individual walks on a floor pressure mat, the system can analyze the 3D shape of the pedal steps and time-based perception data and successfully identify personal information.…”