Towards Standardized Processes for Physical Therapists to Quantify Patient Rehabilitation

Kelliher, Aisling; Zilevu, Setor; Rikakis, Thanassis; Ahmed, Tamim; Truong, Yen Bach; Wolf, Steven L.

doi:10.1145/3313831.3376706

Cited by 6 publications

(24 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The system can be easily installed on a regular kitchen or living room Table . The two cameras initiate and record only during training, and they are activated and controlled by the patient using a customdesigned application on the tablet computer. The system aims to integrate expert knowledge with data driven algorithms to realize coarse real-time automated assessment of movement of stroke survivors during therapy at the home (Kelliher et al, 2020). This assessment can drive high-level feedback on results and performance after execution of each training task, to assist patients with self assessment, and to help them plan their next attempt(s) of the training task.…”

Section: Introductionmentioning

confidence: 99%

“…Our approach has two components: i) making the expert raters process as observable as possible; and ii) leveraging the expert rating process to inform the structuring and improved performance of computational algorithms. In previous publications, we have presented in detail our research and development activities for the first component (Rikakis et al, 2018;Clark and Kelliher, 2021;Kelliher et al, 2020). Inspired by clinical measures for rating rehabilitation movement, we developed the SARAH system to utilize a standardized activity space with eight well defined sub-spaces that are drawn as bounding boxes on the video capture of therapy (see Fig.…”

Section: Introductionmentioning

confidence: 99%

“…We designed the SARAH training objects to facilitate generalized mapping of training tasks to ADLs for the patients, while also facilitating tracking through low-cost video cameras (Kelliher et al, 2019). We used participatory design processes and custom-designed interactive video rating tools to help expert therapists reveal and reflect on their rating process and internalized (tacit) rating schemas (Kelliher et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

“…Even though most therapists use intuitive segmentation of movement for observation and assessment, the segment vocabulary is not standardized. We worked with expert therapists to standardize the segment vocabulary into a state machine that can produce all 15 tasks of the SARAH system (Kelliher et al, 2020). The segments are: Initiation + Progression + Termination (IPT), Manipulate and Transport (MTR), Complex Manipulation and Transport (CMTR), and Release and Return (RR).…”

Section: Introductionmentioning

confidence: 99%

“…This process is not well standardized as the filtering and compositing activities are based on individual experience and training. We further worked with expert therapists to define a consensus-limited set of composite movement features that are important when assessing the performance of each segment in our model Kelliher et al (2020). For example, the resulting rubric identifies four key features to assess during the Complex Manipulation and Transport stage: i) appropriate initial finger positioning, ii) appropriate finger motion after positioning, iii) appropriate limb motion following finger positioning, and iv) limb trajectory with appropriate accuracy.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Automated movement assessment in stroke rehabilitation

Ahmed

Thopalli

Rikakis

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We are developing a system for long-term Semi-Automated Rehabilitation At the Home (SARAH) that relies on low-cost and unobtrusive video-based sensing. We present a cyber-human methodology used by the SARAH system for automated assessment of upper extremity stroke rehabilitation at the home. We propose a hierarchical model for automatically segmenting stroke survivor's movements and generating training task performance assessment scores during rehabilitation. The hierarchical model fuses expert therapist knowledge-based approaches with data-driven techniques. The expert knowledge is more observable in the higher layers of the hierarchy (task and segment) and therefore more accessible to algorithms incorporating high-level constraints relating to activity structure (i.e. type and order of segments per task). We utilize an HMM and a Decision Tree model to connect these high-level priors to data-driven analysis. The lower layers (RGB images and raw kinematics) need to be addressed primarily through data-driven techniques. We use a transformer-based architecture operating on low-level action features (tracking of individual body joints and objects) and a Multi-Stage Temporal Convolutional Network(MS-TCN) operating on raw RGB images. We develop a sequence combining these complementary algorithms effectively, thus encoding the information from different layers of the movement hierarchy. Through this combination, we produce robust segmentation and task assessment results on noisy, variable, and limited data, which is characteristic of low-cost video capture of rehabilitation at the home. Our proposed approach achieves 85% accuracy in per-frame labeling, 99% accuracy in segment classification, and 93% accuracy in task completion assessment. Although the methodology proposed in this paper applies to upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e. lower extremity training for neurological accidents).

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Automated movement assessment in stroke rehabilitation

Ahmed

Thopalli

Rikakis

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

A Hierarchical Bayesian Model for Cyber-Human Assessment of Rehabilitation Movement

Ahmed

Thopali

Rikakis

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Background: The evidence-based quantification of the relation between changes in movement quality and functionality can assist clinicians in achieving more effective structuring or adaptations of therapy. Facilitating this quantification through computational tools can also result in the generation of large-scale data sets that can inform automated assessment of rehabilitation. Interpretable automated assessment can leave more time for clinicians to focus on treatment and allow for remotely supervised therapy at the home. Methods: In our first experiment, we developed a rating process and accompanying computational tool to assist clinicians in following a standardized movement assessment process relating functionality to movement quality. We conducted three studies with three different versions of the computational rating tool. Clinicians rated task, segment, and movement feature performance for 440 videos in which stroke survivors executed standardized upper extremity therapy tasks related to functional activities. In our second experiment, we used the 440 rated videos, in addition to 140 videos of unimpaired subjects performing the same tasks, to improve our previously developed automated assessment ensemble model that automatically generates segmentation times and task ratings across the impaired and unimpaired movement. The automated assessment ensemble integrates expert knowledge constraints into data-driven training through a combination of HMM, transformer, MSTCN++, and decision tree computational modules. In our third experiment, we used the therapist and automated ratings to develop a four-layer Hierarchical Bayesian Model (HBM) for computing the statistical relation of movement quality changes to functionality. We first calculated conditional layer probabilities using clinician ratings of task, segment, and movement features. We increased the granularity of observation of the HBM by formulating ΔHBM, a correlation graph between kinematics and movement composite features. Finally, we used k-means clustering on the ΔHBM to identify three clusters of features among the 16 movement composite and 20 kinematic features and used the centroid of these clusters as the weights of the input data to our computational assessment ensemble. Results: We evaluated the efficacy of our rating interface in terms of inter-rater reliability (IRR) across tasks, segments, and movement features. The third version of the interface produced an average IRR of 67%, while the time per session (TPS) was the lowest of the three studies. By analyzing the ratings, we were able to identify a small number of movement features that have the highest probability of predicting functional improvement. We evaluated the performance of our automated assessment model using 60% impaired and 40% unimpaired movement data and achieved a frame-wise segmentation accuracy of 87.85±0.58 and a block-segmentation accuracy of 98.46±1.6. We also demonstrated the performance of our proposed HBM in correlation to clinician ratings with a correlation of over 90%. The HBM also generates a correlation graph, ΔHBM that relates 16 composite movement features to the 20 kinematic features. We can thus integrate the HBM into the computational assessment ensemble to perform automated and integrated movement quality and functionality assessment that is driven by computationally extracted kinematics. Conclusions: Combining standardized clinician ratings of videos with knowledge based and data-driven computational analysis of rehabilitation movement allows the expression of an HBM that increases the observability of the relation of movement quality to functionality and enables the training of computational algorithms for automated assessment of rehabilitation movement. While our work primarily focuses on the upper extremity of stroke survivors, the models can be adopted to many other neurorehabilitation contexts.

show abstract

Understanding the Needs and Values of Rehabilitation Therapists in Designing and Implementing Telehealth Solutions

Clark

Kelliher

2021

Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Towards Standardized Processes for Physical Therapists to Quantify Patient Rehabilitation

Cited by 6 publications

References 16 publications

Automated movement assessment in stroke rehabilitation

Automated movement assessment in stroke rehabilitation

A Hierarchical Bayesian Model for Cyber-Human Assessment of Rehabilitation Movement

Understanding the Needs and Values of Rehabilitation Therapists in Designing and Implementing Telehealth Solutions

Contact Info

Product

Resources

About