Remote Digital Measurement of Facial and Vocal Markers of Major Depressive Disorder Severity and Treatment Response: A Pilot Study

Abstract

Objectives: Multiple machine learning-based visual and auditory digital markers have demonstrated associations between major depressive disorder (MDD) status and severity. The current study examines if such measurements can quantify response to antidepressant treatment (ADT) with selective serotonin reuptake inhibitors (SSRIs) and serotonin–norepinephrine uptake inhibitors (SNRIs).Methods: Visual and auditory markers were acquired through an automated smartphone task that measures facial, vocal, and head movement characteristics across 4 weeks of treatment (with time points at baseline, 2 weeks, and 4 weeks) on ADT (n = 18). MDD diagnosis was confirmed using the Mini-International Neuropsychiatric Interview (MINI), and the Montgomery–Åsberg Depression Rating Scale (MADRS) was collected concordantly to assess changes in MDD severity.Results: Patient responses to ADT demonstrated clinically and statistically significant changes in the MADRS [F(2, 34) = 51.62, p < 0.0001]. Additionally, patients demonstrated significant increases in multiple digital markers including facial expressivity, head movement, and amount of speech. Finally, patients demonstrated significantly decreased frequency of fear and anger facial expressions.Conclusion: Digital markers associated with MDD demonstrate validity as measures of treatment response.

Multilingual markers of depression in remotely collected speech samples

“…We found no evidence of associations between pause rate and depressive symptoms in any speech tasks or language, in contrast to previous ndings. [6][7][8][9][10] This discrepancy may be due to the statistics (we took account of clustering of repeated measures within individuals) and none of the other studies were conducted in real world settings, meaning the speech collected could also be subject to the observer paradox (27). This is a phenomenon in which the speaking style of a participant changes due to the presence of a researcher or clinician during the recording session.…”

Modern views of machine learning for precision psychiatry

“…As an example, research effort has been dedicated to using computer vision and voice to directly quantify motor activity. Some recent work has demonstrated that digital phenotyping parameters that reflect gross motor activity including speech characteristics (rate of speech, tone) and facial/head movements are associated with suicidal risk, 189 SSRI response in MDD, 251 negative symptomatology in schizophrenia, 191 and Parkinsonian tremor. 252 Such approaches are now being commercialized for all phases of drug development from proof of concept to direct measurement in order to make decisions about ongoing treatment needs.…”

Modern Views of Machine Learning for Precision Psychiatry

“…As an example, research effort has been dedicated to using computer vision and voice to directly quantify motor activity. Some recent work has demonstrated that digital phenotyping parameters that reflect gross motor activity including speech characteristics (rate of speech, tone) and facial/head movements are associated with suicidal risk [241], SSRI response in major depressive disorder [242], negative symptomatology in schizophrenia [243], and Parkinsonian tremor [244]. Such approaches are now being commercialized for all phases of drug development from proof-ofconcept to direct measurement to make decisions about ongoing treatment needs.…”