Leveraging Activity Recognition to Enable Protective Behavior Detection in Continuous Data

Abstract

Protective behavior exhibited by people with chronic pain (CP) during physical activities is very informative to understanding their physical and emotional states. Existing automatic protective behavior detection (PBD) methods rely on pre-segmentation of activities predefined by users. However, in real life, people perform activities casually. Therefore, where those activities present difficulties for people with CP, technology-enabled support should be delivered continuously and automatically adapted to activity type and occurrence of protective behavior. Hence, to facilitate ubiquitous CP management, it becomes critical to enable accurate PBD over continuous data. In this paper, we propose to integrate human activity recognition (HAR) with PBD via a novel hierarchical HAR-PBD architecture comprising graph-convolution and long short-term memory (GC-LSTM) networks, and alleviate class imbalances using a class-balanced focal categorical cross-entropy (CFCC) loss. Through in-depth evaluation of the approach using a CP patients' dataset, we show that the leveraging of HAR, GC-LSTM networks, and CFCC loss leads to clear increase in PBD performance against the baseline (macro F1 score of 0.81 vs. 0.66 and precision-recall area-under-the-curve (PR-AUC) of 0.60 vs. 0.44). We conclude by discussing possible use cases of the hierarchical architecture in CP management and beyond. We also discuss current limitations and ways forward.

EmoPain(at)Home: Dataset and Automatic Assessment within Functional Activity for Chronic Pain Rehabilitation

“…We captured movement data for the right elbow and wrist, mid spine, hip, and right knee and ankle. Findings in previous work [19], [20], [36] suggest Tidying up room -Note that the participant with PID=3 originally specified 'Washing up' as non-challenging, but referred to it as challenging for a different session when they experienced (chronic) hand pain. PID -Participant ID * -performed without the researcher (remotely) present ** -performed both with and without the researcher (remotely) present that capture of data from only one side of the body can be informative for automatic assessment.…”

Stepping Into the Next Decade of Ubiquitous and Pervasive Computing: UbiComp and ISWC 2021

“…The session chair, Thomas Ploetz, noted the long, challenging process of collecting data and overcoming technical challenges in completing the work. Wang et al 41 designed a novel HARprotective behavior detection (PBD) architecture that recognizes the behavior context to apply the optimal PBD method to serve people with chronic pain during physical activities. Hiremath et al 42 augmented HAR practitioner's ability to understand HAR task complexity through an objective task complexity assessment.…”

Comparison of Deep Learning Approaches for Protective Behaviour Detection Under Class Imbalance from MoCap and EMG data

“…We shall not further describe the state of the art of GCNs for skeleton based action recognition, but Song et al [6] already present a very comprehensive literature review and comparison. GCNs have already been applied to the EMOPAIN dataset as well [12].…”