A wearable vision-based system for detecting hand-object interactions in individuals with cervical spinal cord injury: First results in the home environment

Abstract

Cervical spinal cord injury (cSCI) causes the paralysis of upper and lower limbs and trunk, significantly reducing quality of life and community participation of the affected individuals. The functional use of the upper limbs is the top recovery priority of people with cSCI and wearable vision-based systems have recently been proposed to extract objective outcome measures that reflect hand function in a natural context. However, previous studies were conducted in a controlled environment and may not be indicative of the actual hand use of people with cSCI living in the community. Thus, we propose a deep learning algorithm for automatically detecting hand-object interactions in egocentric videos recorded by participants with cSCI during their daily activities at home. The proposed approach is able to detect hand-object interactions with good accuracy (F1-score up to 0.82), demonstrating the feasibility of this system in uncontrolled situations (e.g., unscripted activities and variable illumination). This result paves the way for the development of an automated tool for measuring hand function in people with cSCI living in the community. Clinical relevance-The accurate detection of hand-object interactions in people with cSCI will allow extracting outcome measures of upper limb function that can be used for planning interventions and tracking the rehabilitation progress remotely.

Grasp Analysis in the Home Environment as a Measure of Hand Function After Cervical Spinal Cord Injury

“…Hand evaluation in clinics only captures a subset of actual hand use in daily activities. Combining egocentric video with advanced machine learning algorithms [15][16][17]19 may allow analyzing other aspects of hand function, such as hand kinematics during reaching to grasping, or complexity of hand posture during transitory to stable grasp phases, to name but a few. These new metrics may be complementary to the information provided by a brief hand assessment in clinics.…”

Perspectives and recommendations of individuals with tetraplegia regarding wearable cameras for monitoring hand function at home: Insights from a community-based study

“…Recent work demonstrated the ability of computer vision to track the user’s hands, 22 detect functional hand-object interactions, 23 , 26 , 27 and identify the presence of tenodesis grasp. 21 These approaches constitute the basis for producing new strategies to monitor rehabilitation progress in people with cSCI living in the community and reporting novel outcome measures of UL function.…”

Application of Video-Based Deep Learning for Early Diagnosis of Neurological Disorders

“…Gait analysis, which is a key aspect of clinical assessments for quantifying functional outcomes following a neurological or musculoskeletal disease, was attempted with DL analyses in various health conditions, such as stroke, Parkinson disease, CP, and spinal cord injury. [2][3][4] In Parkinson disease, DL-based assessments of gait features using video could allow early discovery of gait anomalies in the course of the disease. 4 This solution could be more reliable than commonly used clinical tests, such as the Hoehn and Yahr scale or the Unified Parkinson Disease Rating Scale.…”

“…This interesting approach builds on the efforts of other researchers striving for more accurate detection and evaluation methods for neurological disorders with the help of machine learning. Gait analysis, which is a key aspect of clinical assessments for quantifying functional outcomes following a neurological or musculoskeletal disease, was attempted with DL analyses in various health conditions, such as stroke, Parkinson disease, CP, and spinal cord injury . In Parkinson disease, DL-based assessments of gait features using video could allow early discovery of gait anomalies in the course of the disease .…”

“…Similarly, single-camera videos and pose estimation models based on DL networks could be used to quantify clinically relevant gait metrics in individuals poststroke, such as measuring reduced walking speeds, decreased cadence, prolonged swing time, and reduced stance time on the paretic side . Likewise, a study by Bandini et al proposed a DL algorithm for automatically detecting hand-object interactions in egocentric videos recorded by participants with cervical spinal cord injury during their daily activities at home. This technique allows the assessment of neurological function in an everyday life environment, which may better indicate the actual hand function.…”