Computer Vision-Based Assessment of Motor Functioning in Schizophrenia: Use of Smartphones for Remote Measurement of Schizophrenia Symptomatology

Abstract

<b><i>Introduction:</i></b> Motor abnormalities have been shown to be a distinct component of schizophrenia symptomatology. However, objective and scalable methods for assessment of motor functioning in schizophrenia are lacking. Advancements in machine learning-based digital tools have allowed for automated and remote “digital phenotyping” of disease symptomatology. Here, we assess the performance of a computer vision-based assessment of motor functioning as a characteristic of schizophrenia using video data collected remotely through smartphones. <b><i>Methods:</i></b> Eighteen patients with schizophrenia and 9 healthy controls were asked to remotely participate in smartphone-based assessments daily for 14 days. Video recorded from the smartphone front-facing camera during these assessments was used to quantify the Euclidean distance of head movement between frames through a pretrained computer vision model. The ability of head movement measurements to distinguish between patients and healthy controls as well as their relationship to schizophrenia symptom severity as measured through traditional clinical scores was assessed. <b><i>Results:</i></b> The rate of head movement in participants with schizophrenia (1.48 mm/frame) and those without differed significantly (2.50 mm/frame; <i>p</i> = 0.01), and a logistic regression demonstrated that head movement was a significant predictor of schizophrenia diagnosis (<i>p</i> = 0.02). Linear regression between head movement and clinical scores of schizophrenia showed that head movement has a negative relationship with schizophrenia symptom severity (<i>p</i> = 0.04), primarily with negative symptoms of schizophrenia. <b><i>Conclusions:</i></b> Remote, smartphone-based assessments were able to capture meaningful visual behavior for computer vision-based objective measurement of head movement. The measurements of head movement acquired were able to accurately classify schizophrenia diagnosis and quantify symptom severity in patients with schizophrenia.

Automatic Schizophrenia Detection Using Multimodality Media via a Text Reading Task

“…In recent years, in the research on the analysis of abnormal speech (Holmlund et al, 2020 ; Tang et al, 2021 ) and head movement (Robson et al, 2016 ; Abbas et al, 2021 ) of schizophrenia, the sample number of schizophrenic patients is about 20: Tang et al ( 2021 ) use the dataset of 20 patients vs. 11 controls; Holmlund et al ( 2020 ) use the dataset of 25 patients vs. 79 controls; Abbas et al ( 2021 ) use the dataset of 18 patients vs. 9 controls; Robson et al ( 2016 ) use the dataset of 23 patients vs. 23 controls.…”

“…Head movement is mainly body movement when reading the specified text in a fixed position. Studies have reported that schizophrenic patients have significant differences in head movement rates from controls (Abbas et al, 2021 ), and produce unconscious and unnecessary head movement in visual tasks (Yoo et al, 2005 ). The head movement measuring methods used in the present research include motion energy analysis (MEA) (Kupper et al, 2010 , 2015 ) and head-mounted motion sensors (Leask et al, 2012 ).…”

Remote Digital Measurement of Facial and Vocal Markers of Major Depressive Disorder Severity and Treatment Response: A Pilot Study

“…This software platform has historically been used in clinical research for reporting of patient behavior to clinicians, including medication adherence, electronic patient-reported outcomes, and ecological momentary assessments, with considerable work done on patient acceptance and usability ( 32 , 33 ). An additional functionality of capturing video and audio in response to prompts (as described below) was utilized for the purposes of this study ( 34 , 35 ).…”

“…Montgomery-Åsberg Depression Rating Scale: The MADRS is a 10-item clinician administered scale for the measurement of MDD with validated clinical cut points for severe (>34), moderate (20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30)(31)(32)(33)(34), mild (7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17)(18)(19), and asymptomatic (<7) depression. The MADRS has demonstrated validity as a sensitive measure of ADT response (31).…”

Modern Views of Machine Learning for Precision Psychiatry

“…Although body language and facial expressions form a key part of a psychiatric exam, ML has only recently been applied to analyze such data objectively. To date, most work has been targeted to support the care for suicidal ideation [187] and depression [165], [166], [188], with few studies also dedicated to schizophrenia [175] and autism spectrum disorders [189]. Features derived from video data have been found to be relevant across all of these disorders.…”