Deep Learning for Monitoring of Human Gait: A Review

Abstract

The essential human gait parameters are briefly reviewed, followed by a detailed review of the state of the art in deep learning for the human gait analysis. The modalities for capturing the gait data are grouped according to the sensing technology: video sequences, wearable sensors, and floor sensors, as well as the publicly available datasets. The established artificial neural network architectures for deep learning are reviewed for each group, and their performance are compared with particular emphasis on the spatiotemporal character of gait data and the motivation for multi-sensor, multi-modality fusion. It is shown that by most of the essential metrics, deep learning convolutional neural networks typically outperform shallow learning models. In the light of the discussed character of gait data, this is attributed to the possibility to extract the gait features automatically in deep learning as opposed to the shallow learning from the handcrafted gait features.

Sensor Fusion for Analysis of Gait under Cognitive Load: Deep Learning Approach

“…The iMAGiMAT system captures a sequence of periodic events characterized as repetitive cycles for each foot. Further explanation of gait cycle events can be found in [15]. Figure 4 shows the spatial average SA of the spatiotemporal gait signal based on the GRF:…”

“…The LRP relevance scores point out which parts of the gait spatiotemporal signal were most relevant for classification, as shown in figure 7 for three subjects. To understand this approach, we note that on multiple repetitive occasions each subject will initiate a gait cycle (explained in [15]) by heel strike, strictly followed by other gait events described in figure 4. Figure 4 details a subject's recorded gait temporal signal, where it starts by heel strike and ends by toe off when the user steps out of the iMAGiMAT floor sensor.…”

“…It is worth mentioning that the results reported at this point, and the LRP decomposition of gait classifications in particular, indicate a possible bridge between the output of artificial intelligence systems for processing of high quality gait data and decisions based on visual observations by humans, or quantitative parameters derived from such observations, (c.f. figure 1 in [15]) which are tested and routinely implemented in current practice. In healthcare alone, these approaches may address some of the significant interest in the detection of the onset of Parkinson disease [24] and Alzheimer's [25], freezing of gait [26], and fall risks [8].…”

“…The architecture of the CNN model engineered for this study is shown in figure 2. The model is proposed based on extensive research on gait in our previous work [14], [15], [16], [17]. The model consists of 12 stacked layers to .…”

Deep Learning and Sensor Fusion Methods for Studying Gait Changes Under Cognitive Load in Males and Females

“…(1)) to show the temporal activity on the surface of the carpet. As shown further, the processing of the calculated temporal GRF signal, allows interpretation as to which gait event occurred on the carpet surface, using gait cycle descriptors [12].…”