Facial and Vocal Markers of Schizophrenia Measured Using Remote Smartphone Assessments: Observational Study

Abstract

Background Machine learning–based facial and vocal measurements have demonstrated relationships with schizophrenia diagnosis and severity. Demonstrating utility and validity of remote and automated assessments conducted outside of controlled experimental or clinical settings can facilitate scaling such measurement tools to aid in risk assessment and tracking of treatment response in populations that are difficult to engage. Objective This study aimed to determine the accuracy of machine learning–based facial and vocal measurements acquired through automated assessments conducted remotely through smartphones. Methods Measurements of facial and vocal characteristics including facial expressivity, vocal acoustics, and speech prevalence were assessed in 20 patients with schizophrenia over the course of 2 weeks in response to two classes of prompts previously utilized in experimental laboratory assessments: evoked prompts, where subjects are guided to produce specific facial expressions and speech; and spontaneous prompts, where subjects are presented stimuli in the form of emotionally evocative imagery and asked to freely respond. Facial and vocal measurements were assessed in relation to schizophrenia symptom severity using the Positive and Negative Syndrome Scale. Results Vocal markers including speech prevalence, vocal jitter, fundamental frequency, and vocal intensity demonstrated specificity as markers of negative symptom severity, while measurement of facial expressivity demonstrated itself as a robust marker of overall schizophrenia symptom severity. Conclusions Established facial and vocal measurements, collected remotely in schizophrenia patients via smartphones in response to automated task prompts, demonstrated accuracy as markers of schizophrenia symptom severity. Clinical implications are discussed.

Sentiment Analysis on Social Media

“…[4] The latest type of information unfairness comes from Application Programming Interface(API) ambush that works on both algorithms, information/data, and substantiate outcomes. [5] This type of work helps us to understand how different types of information, ideas and characteristics of social media information/data can be largely considered and authenticated and in what way can be in touch arbitration mechanisms can be drafted to control pessimistic consequences.…”

Modern views of machine learning for precision psychiatry

“… 169 Posture head pitch variance, upper body movements reduced head movement in schizophrenia Abbas et al. 191 higher head movement in ASD de Belen et al. 192 Language Grandiosity unrealistic sense of superiority increased in bipolar Castro et al.…”

“…As an example, research effort has been dedicated to using computer vision and voice to directly quantify motor activity. Some recent work has demonstrated that digital phenotyping parameters that reflect gross motor activity including speech characteristics (rate of speech, tone) and facial/head movements are associated with suicidal risk, 189 SSRI response in MDD, 251 negative symptomatology in schizophrenia, 191 and Parkinsonian tremor. 252 Such approaches are now being commercialized for all phases of drug development from proof of concept to direct measurement in order to make decisions about ongoing treatment needs.…”

Modern Views of Machine Learning for Precision Psychiatry

“…As an example, research effort has been dedicated to using computer vision and voice to directly quantify motor activity. Some recent work has demonstrated that digital phenotyping parameters that reflect gross motor activity including speech characteristics (rate of speech, tone) and facial/head movements are associated with suicidal risk [241], SSRI response in major depressive disorder [242], negative symptomatology in schizophrenia [243], and Parkinsonian tremor [244]. Such approaches are now being commercialized for all phases of drug development from proof-ofconcept to direct measurement to make decisions about ongoing treatment needs.…”