Component-Level Tuning of Kinematic Features From Composite Therapist Impressions of Movement Quality

Venkataraman, Vinay; Turaga, Pavan; Baran, Michael; Lehrer, Nicole; Du, Tingfang; Cheng, Long; Rikakis, Thanassis; Wolf, Steven L.

doi:10.1109/jbhi.2014.2375206

Cited by 9 publications

(7 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We experimented with 8, 4, 2, and 1 keypoints from open pose combined with object movement features. We selected the keypoints representing joints that have been show, in our previous work, to have prominence in characterizing upper limb impaired movement ( 32 , 34 ). It can thus be said that some form of prior expert knowledge is used to constrain the Transformer.…”

Section: Discussionmentioning

confidence: 99%

“…The proposed hierarchy aims to reveal the grammar (or compositional structure) of therapy, from the signal level to the meaningful activity level (training tasks). Knowing the composition rules of a complex human activity, whether that is language, sport, or a board game like chess, facilitates the meaningful computational analysis of the activity in a manner that is comprehended and leveraged by expert trainers and trainees ( 33 , 34 ). Although the hierarchy proposed in this paper is established for upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e., lower extremity training for neurological accidents).…”

Section: Methodsmentioning

confidence: 99%

“…Knowing the composition rules of a complex human activity, whether that is language, sport, or a board game like FIGURE 3 | Hierarchical representation of the cyber-human system; (from the bottom) a video frame as an example of raw visual features, velocity profile as an example of raw kinematic features, composite feature for task 5 generated using PCA, segment labels, different types of tasks and impairment levels. chess, facilitates the meaningful computational analysis of the activity in a manner that is comprehended and leveraged by expert trainers and trainees (33,34). Although the hierarchy proposed in this paper is established for upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e., lower extremity training for neurological accidents).…”

Section: Analysis Frameworkmentioning

confidence: 99%

See 2 more Smart Citations

Automated Movement Assessment in Stroke Rehabilitation

et al. 2021

Self Cite

View full text Add to dashboard Cite

We are developing a system for long term Semi-Automated Rehabilitation At the Home (SARAH) that relies on low-cost and unobtrusive video-based sensing. We present a cyber-human methodology used by the SARAH system for automated assessment of upper extremity stroke rehabilitation at the home. We propose a hierarchical model for automatically segmenting stroke survivor's movements and generating training task performance assessment scores during rehabilitation. The hierarchical model fuses expert therapist knowledge-based approaches with data-driven techniques. The expert knowledge is more observable in the higher layers of the hierarchy (task and segment) and therefore more accessible to algorithms incorporating high level constraints relating to activity structure (i.e., type and order of segments per task). We utilize an HMM and a Decision Tree model to connect these high level priors to data driven analysis. The lower layers (RGB images and raw kinematics) need to be addressed primarily through data driven techniques. We use a transformer based architecture operating on low-level action features (tracking of individual body joints and objects) and a Multi-Stage Temporal Convolutional Network(MS-TCN) operating on raw RGB images. We develop a sequence combining these complimentary algorithms effectively, thus encoding the information from different layers of the movement hierarchy. Through this combination, we produce a robust segmentation and task assessment results on noisy, variable and limited data, which is characteristic of low cost video capture of rehabilitation at the home. Our proposed approach achieves 85% accuracy in per-frame labeling, 99% accuracy in segment classification and 93% accuracy in task completion assessment. Although the methodology proposed in this paper applies to upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e., lower extremity training for neurological accidents).

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Analysis Frameworkmentioning

confidence: 99%

See 1 more Smart Citation

Automated Movement Assessment in Stroke Rehabilitation

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…We experimented with 8, 4, 2 and 1 keypoints from open pose combined with object movement features. We selected the keypoints representing joints that have been show, in our previous work, to have prominence in characterizing upper limb impaired movement (Venkataraman et al, 2014(Venkataraman et al, , 2016. It can thus be said that some form of prior expert knowledge is used to constrain the Transformer.…”

Section: Discussionmentioning

confidence: 99%

“…The proposed hierarchy aims to reveal the grammar (or compositional structure) of therapy, from the signal level to the meaningful activity level (training tasks). Knowing the composition rules of a complex human activity, whether that is language, sport, or a board game like chess, facilitates the meaningful computational analysis of the activity in a manner that is comprehended and leveraged by expert trainers and trainees Simon (1981); Venkataraman et al (2016). Although the hierarchy proposed in this paper is established for upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e.…”

Section: Analysis Frameworkmentioning

confidence: 99%

Automated movement assessment in stroke rehabilitation

Ahmed

Thopalli

Rikakis

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We are developing a system for long-term Semi-Automated Rehabilitation At the Home (SARAH) that relies on low-cost and unobtrusive video-based sensing. We present a cyber-human methodology used by the SARAH system for automated assessment of upper extremity stroke rehabilitation at the home. We propose a hierarchical model for automatically segmenting stroke survivor's movements and generating training task performance assessment scores during rehabilitation. The hierarchical model fuses expert therapist knowledge-based approaches with data-driven techniques. The expert knowledge is more observable in the higher layers of the hierarchy (task and segment) and therefore more accessible to algorithms incorporating high-level constraints relating to activity structure (i.e. type and order of segments per task). We utilize an HMM and a Decision Tree model to connect these high-level priors to data-driven analysis. The lower layers (RGB images and raw kinematics) need to be addressed primarily through data-driven techniques. We use a transformer-based architecture operating on low-level action features (tracking of individual body joints and objects) and a Multi-Stage Temporal Convolutional Network(MS-TCN) operating on raw RGB images. We develop a sequence combining these complementary algorithms effectively, thus encoding the information from different layers of the movement hierarchy. Through this combination, we produce robust segmentation and task assessment results on noisy, variable, and limited data, which is characteristic of low-cost video capture of rehabilitation at the home. Our proposed approach achieves 85% accuracy in per-frame labeling, 99% accuracy in segment classification, and 93% accuracy in task completion assessment. Although the methodology proposed in this paper applies to upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e. lower extremity training for neurological accidents).

show abstract