Visual perceptive deep learning for smartphone video-based tremor analysis: VIPER-Tremor

Abstract

Background: Tremor is one of the most common neurological symptoms. Its clinical and neurobiological complexity necessitates novel approaches for deep and granular phenotyping. Instrumented neurophysiological analyses have proven useful for clinical management, but are highly resource-intensive and lack broad accessibility. Simplified bedside scores, on the other hand, lack the granularity to capture subtle but relevant tremor features. Addressing this gap, we develop a deep learning framework for the quantitative assessment of limb tremor utilizing only standard clinical videos. Methods: We engineer a visual perceptive limb tremor analysis tool based on Mediapipe, a convolutional neural network architecture for marker-less hand tracking: VIPER-Tremor. We validate it against gold standard methods, including marker-based motion capture, wrist-worn accelerometery, and clinical scoring across two independent clinical cohorts encompassing a total of 66 patients diagnosed with essential tremor and recorded in different therapeutic states of deep brain stimulation. Results: Computer vision-derived tremor metrics exhibit high convergent clinical validity to scores (Spearman’s rho= 0.55 – 0.86, p≤ .01) as well as an accuracy of up to 2.60mm and ≤0.21Hz for tremor amplitude and frequency measurements, matching gold-standard equipment. VIPER-Tremor is capable of extracting advanced tremor features relevant for differential diagnosis and enables therapeutic outcome prediction, a dimension which conventional tremor scores were unable to provide. Conclusion: VIPER-Tremor is an accurate, unbiased and highly accessible solution for smartphone video-based tremor analysis and yields comparable results to gold standard recordings. VIPER-Tremor presents a significant advancement in tremor analysis, combining accuracy and accessibility, and promises to be a pivotal tool in the emerging field of precision neurology, enhancing diagnostic and therapeutic approaches.

Development of a Tremor Detection Algorithm for use in an Academic Movement Disorders Center

“…Further, our testing approach imposes few, if any constraints on the participants’ natural movements. This complicates data analysis compared to methods that confine movements to a single plane, (e.g., [13]) but may improve external validity.…”

“…Despite this, technologies for identifying tremor have progressed towards standardization and clinical uptake very slowly [1]. In the research domain, various technologies measure human motion, including body-worn sensors [3,7,8], 3D motion capture [9,10], and -most recently -pose recognition from monocular video [11][12][13]. Digitizing tablets are often used for assessing tremor during tasks like spiral drawing [14,15] and for discriminating tremor from bradykinesia during finger tapping [16].…”