Improving Inertial Sensor-Based Activity Recognition in Neurological Populations

Abstract

Inertial sensor-based human activity recognition (HAR) has a range of healthcare applications as it can indicate the overall health status or functional capabilities of people with impaired mobility. Typically, artificial intelligence models achieve high recognition accuracies when trained with rich and diverse inertial datasets. However, obtaining such datasets may not be feasible in neurological populations due to, e.g., impaired patient mobility to perform many daily activities. This study proposes a novel framework to overcome the challenge of creating rich and diverse datasets for HAR in neurological populations. The framework produces images from numerical inertial time-series data (initial state) and then artificially augments the number of produced images (enhanced state) to achieve a larger dataset. Here, we used convolutional neural network (CNN) architectures by utilizing image input. In addition, CNN enables transfer learning which enables limited datasets to benefit from models that are trained with big data. Initially, two benchmarked public datasets were used to verify the framework. Afterward, the approach was tested in limited local datasets of healthy subjects (HS), Parkinson’s disease (PD) population, and stroke survivors (SS) to further investigate validity. The experimental results show that when data augmentation is applied, recognition accuracies have been increased in HS, SS, and PD by 25.6%, 21.4%, and 5.8%, respectively, compared to the no data augmentation state. In addition, data augmentation contributes to better detection of stair ascent and stair descent by 39.1% and 18.0%, respectively, in limited local datasets. Findings also suggest that CNN architectures that have a small number of deep layers can achieve high accuracy. The implication of this study has the potential to reduce the burden on participants and researchers where limited datasets are accrued.

Data Augmentation Techniques for Accurate Action Classification in Stroke Patients with Hemiparesis

“…A few studies [ 12 , 13 ] have adopted an approach similar to that used in this study. Um et al [ 12 ] applied various time-series transformations to augment movement data related to Parkinson’s disease to enhance a binary classifier’s performance in distinguishing bradykinesia and dyskinesia.…”

“…Um et al [ 12 ] applied various time-series transformations to augment movement data related to Parkinson’s disease to enhance a binary classifier’s performance in distinguishing bradykinesia and dyskinesia. Celik et al [ 13 ] converted IMU data into images and subjected them to data augmentation to improve the classification performance of pre-trained CNN models. However, these studies tested data augmentation under restricted conditions, with conclusions that may not be generalizable to a broader context.…”

“…It demonstrated that most transformations enhance the classification performance of a convolutional neural network (CNN) model, with rotation, permutation, and time-warping proving to be the most effective. Celik et al [ 13 ] converted IMU data into images and augmented to improve the classification performance of pre-trained CNN models. The data consisted of six classes (walking, ascending, descending, sitting, standing, and lying down) sourced from healthy subjects, individuals with Parkinson’s, and stroke survivors.…”

A Novel Framework Based on Deep Learning Architecture for Continuous Human Activity Recognition with Inertial Sensors

“…The scientific literature has given more and more attention to the field of human activity recognition (HAR), which aims to classify human actions by exploiting sensor data [7]. HAR has covered various contexts, from industry [8,9] to sport [10], but a wider application lies in the medical field [7,[10][11][12][13][14]: in this realm, subjects' activities can be remotely registered outside the clinic [15] and clinicians can evaluate their functional abilities after treatment [16,17]. HAR can also enhance a rehabilitative program inside the clinic for the sake of an assist-as-needed approach: in particular, recognizing the motor actions performed by patients (e.g., post-stroke individuals or people with psychomotor dysfunction) can allow for correcting motions or encouraging further exercise when required [18].…”

“…A typical HAR experimental protocol encompasses a set of activities that the subject is asked to perform. These motor tasks may involve mainly upper body [9,20], or lower body (e.g., walking or climbing/descending stairs) [15,17,21,22], or even require the individual to drive upper-and lower-extremities in a proper combination (e.g., lying in bed) [23][24][25][26][27][28][29]. Furthermore, the protocol to collect data for HAR purposes tends to be designed such that ADLs are performed separately, i.e., a batch of repetitions of one of the activities to be recognized is asked to be executed by every subject [30].…”