Unobtrusive Inference of Affective States in Virtual Rehabilitation from Upper Limb Motions: A Feasibility Study

Abstract

Virtual rehabilitation environments may afford greater patient personalization if they could harness the patient's affective state. Four states: anxiety, pain, engagement and tiredness (either physical or psychological), were hypothesized to be inferable from observable metrics of hand location and gripping strength-relevant for rehabilitation-. Contributions are; (a) multiresolution classifier built from Semi-Naïve Bayesian classifiers, and (b) establishing predictive relations for the considered states from the motor proxies capitalizing on the proposed classifier with recognition levels sufficient for exploitation. 3D hand locations and gripping strength streams were recorded from 5 post-stroke patients whilst undergoing motor rehabilitation therapy administered through virtual rehabilitation along 10 sessions over 4 weeks. Features from the streams characterized the motor dynamics, while spontaneous manifestations of the states were labelled from concomitant videos by experts for supervised classification. The new classifier was compared against baseline support vector machine (SVM) and random forest (RF) with all three exhibiting comparable performances. Inference of the aforementioned states departing from chosen motor surrogates appears feasible, expediting increased personalization of virtual motor neurorehabilitation therapies.

Automatic Recognition of Multiple Affective States in Virtual Rehabilitation by Exploiting the Dependency Relationships

“…A dataset of post-stroke patients [11] was used to assess the performance of the proposed computational models which exploits the dependency relationships between the patient's affective states. This dataset contains the performance records of 5 post-stroke patients while using the GT system during ten sessions over a period of about one month (each session was taken in a different day, maximum 3 sessions per week).…”

“…Our proposal uses as base classifier a derivation from Naive Bayes classifier, named Semi-Naive Bayesian classifier (SNBC) [9], for its efficiency, simplicity and because it tackles dependent features [10]. SNBC is used to build the Multiresolution SNBC (MSNB) [11] and then to create the late Fusion of the three sensors (PRE, MOV, and FAE) using SNBC (FSNB). Finally, the states dependency relationships are exploited by a multi-label classifier named circular classifier chains (CCC) [12].…”

“…For PRE and MOV, FSNB achieved over 82% in AUC, except for pain were the recognition rates were around 70% in AUC. Our Bayesian approach of MSNB classifier was compared against SVM and random forest [11], and the results were promising. An advantage of the Bayesian approaches is that their models are interpretable.…”

Multiple Instance Learning for Emotion Recognition Using Physiological Signals

“…As affective computing continues to mature, naturalistic datasets are becoming available. Some of these datasets are being fully labeled manually by experts [3], [7], [12] or naive raters (AVEC, [8]- [10], [13]), and different forms of inter-rater reliability have been used. Despite the importance that manual labeling has had in the field of machine learning, in many cases, such labeling is not feasible or scalable.…”

“…This could create frustrating disruptions to people's tasks (e.g. [12]) and in the case of patients it may even lead to unhealthy catastrophising on one's condition [18].…”