Improving Accelerometry-Based Measurement of Functional Use of the Upper Extremity After Stroke: Machine Learning Versus Counts Threshold Method

Abstract

Background Wrist-worn accelerometry provides objective monitoring of upper-extremity functional use, such as reaching tasks, but also detects nonfunctional movements, leading to ambiguity in monitoring results. Objective Compare machine learning algorithms with standard methods (counts ratio) to improve accuracy in detecting functional activity. Methods Healthy controls and individuals with stroke performed unstructured tasks in a simulated community environment (Test duration = 26 ± 8 minutes) while accelerometry and video were synchronously recorded. Human annotators scored each frame of the video as being functional or nonfunctional activity, providing ground truth. Several machine learning algorithms were developed to separate functional from nonfunctional activity in the accelerometer data. We also calculated the counts ratio, which uses a thresholding scheme to calculate the duration of activity in the paretic limb normalized by the less-affected limb. Results The counts ratio was not significantly correlated with ground truth and had large errors ( r = 0.48; P = .16; average error = 52.7%) because of high levels of nonfunctional movement in the paretic limb. Counts did not increase with increased functional movement. The best-performing intrasubject machine learning algorithm had an accuracy of 92.6% in the paretic limb of stroke patients, and the correlation with ground truth was r = 0.99 ( P < .001; average error = 3.9%). The best intersubject model had an accuracy of 74.2% and a correlation of r =0.81 ( P = .005; average error = 5.2%) with ground truth. Conclusions In our sample, the counts ratio did not accurately reflect functional activity. Machine learning algorithms were more accurate, and future work should focus on the development of a clinical tool.

A Framework for Sensor-based Assessment of Upper-limb Functioning

“…Both these methods have relatively poor accuracy in detecting meaningful movements/postures due to poor specificity or sensitivity [24]. Recent work on machine learning based methods [15, 16] have demonstrated better performance in detecting upper-limb use than existing methods. Future investigation into more sophisticated methods and the availability of more data is likely to improve this performance in upper-limb use detection.…”

“…al [15] proposed the use of a random forests classifier to detect upper-limb use from features extracted from an accelerometer. The ML approach can be used with both accelerometers and IMUs, and has reasonable sensitivity and specificity [16].…”

“…There is growing interest in wearable sensors for continuous and objective monitoring of upper-limb functioning [8–16]. When developing a sensor-based assessment tool, there are a four major interdependent design choices that influence the nature of the information conveyed by the assessment: (a) the type of sensing modality used for measurements (e.g., camera-based movement tracking, inertial measurement units), (b) steps involved in the data processing pipeline (e.g., data segmentation, filtering), (c) properties of measures used to quantify constructs of interest (e.g., sensitivity to movement changes in the physiological changes, robustness to measurement noise), and (d) the nature of data visualization methods employed (e.g., temporal evolution of the measure, scatter plots of different variables).…”

“…Inertial sensors composed of accelerometers and gyroscopes have been the preferred modality for assessing upper-limb functioning in the natural setting, due to their availability, affordability, and compact size [8–16]. Thus far, the focus of sensor-based assessment in hemiparesis has been the quantification of the overall amount (question 1) and the relative bias (question 2) in using the arms during daily life [8–16]. The current methods for quantifying the amount of upper-limb use have either used: (a) the magnitude of acceleration (e.g.…”

“…activity counting (AC) [8, 11, 20]) or (b) the duration of functional movements detected from sensor data (e.g. gross movement (GM) score [10, 13], machine learning (ML) algorithms [14–16]). Although related, movement duration and intensity convey slightly different information about the nature of arm use.…”