Activity Recognition in Individuals Walking With Assistive Devices: The Benefits of Device-Specific Models

Abstract

BackgroundWearable sensors gather data that machine-learning models can convert into an identification of physical activities, a clinically relevant outcome measure. However, when individuals with disabilities upgrade to a new walking assistive device, their gait patterns can change, which could affect the accuracy of activity recognition.ObjectiveThe objective of this study was to assess whether we need to train an activity recognition model with labeled data from activities performed with the new assistive device, rather than data from the original device or from healthy individuals.MethodsData were collected from 11 healthy controls as well as from 11 age-matched individuals with disabilities who used a standard stance control knee-ankle-foot orthosis (KAFO), and then a computer-controlled adaptive KAFO (Ottobock C-Brace). All subjects performed a structured set of functional activities while wearing an accelerometer on their waist, and random forest classifiers were used as activity classification models. We examined both global models, which are trained on other subjects (healthy or disabled individuals), and personal models, which are trained and tested on the same subject.ResultsMedian accuracies of global and personal models trained with data from the new KAFO were significantly higher (61% and 76%, respectively) than those of models that use data from the original KAFO (55% and 66%, respectively) (Wilcoxon signed-rank test, P=.006 and P=.01). These models also massively outperformed a global model trained on healthy subjects, which only achieved a median accuracy of 53%. Device-specific models conferred a major advantage for activity recognition.ConclusionsOur results suggest that when patients use a new assistive device, labeled data from activities performed with the specific device are needed for maximal precision activity recognition. Personal device-specific models yield the highest accuracy in such scenarios, whereas models trained on healthy individuals perform poorly and should not be used in patient populations.

The Effect of Sensor Placement and Number on Physical Activity Recognition and Energy Expenditure Estimation in Older Adults: Validation Study

“…Although this sample had common comorbidities such as diabetes, hypertension, and cancer history, we did not actively recruit people who had specific ambulatory deficits that would likely impact the results. Existing work in these specialized populations shows that knowledge from nonambulatory, impaired (eg, healthier) adults transfers with poor accuracy [ 55 ]. Thus, this study is limited to community-dwelling older adults without overt ambulatory deficits.…”

AI in Rehabilitation Medicine: Opportunities and Challenges

“…These measurements should be validated against the "gold standard" measurement technique to assess their accuracy, reliability, and agreement level [69][70][71]. Measurement validation should, as best as possible, include the expected environmental conditions and patients that are representative of the model's expected use case [48,72,73]. Protocols for measurement validation should be reported in detail to increase the later AI model's transparency to potential sources of error.…”

“…Regression algorithms are used to estimate a function between input features and continuous outputs like time to hospital discharge or clinical score [85]. Classification algorithms identify discrete quantities, such as activities [48,72], or impairment and disease categories [38,40,86,87]. Deep learning models, including neural networks, directly learn patterns from data via reinforcement techniques [88].…”

Re-Enactment as a Method to Reproduce Real-World Fall Events Using Inertial Sensor Data: Development and Usability Study

“…However, there is still an unacceptable high false alarm rate. Machine-learning approaches already show promising results in activity recognition based on data from waist-worn inertial sensors [26,27] and might further improve the results, but would need additional realistic fall data.…”