Validation of Visual and Auditory Digital Markers of Suicidality in Acutely Suicidal Psychiatric Inpatients: Proof-of-Concept Study

Abstract

Background Multiple symptoms of suicide risk have been assessed based on visual and auditory information, including flattened affect, reduced movement, and slowed speech. Objective quantification of such symptomatology from novel data sources can increase the sensitivity, scalability, and timeliness of suicide risk assessment. Objective We aimed to examine measurements extracted from video interviews using open-source deep learning algorithms to quantify facial, vocal, and movement behaviors in relation to suicide risk severity in recently admitted patients following a suicide attempt. Methods We utilized video to quantify facial, vocal, and movement markers associated with mood, emotion, and motor functioning from a structured clinical conversation in 20 patients admitted to a psychiatric hospital following a suicide risk attempt. Measures were calculated using open-source deep learning algorithms for processing facial expressivity, head movement, and vocal characteristics. Derived digital measures of flattened affect, reduced movement, and slowed speech were compared to suicide risk with the Beck Scale for Suicide Ideation controlling for age and sex, using multiple linear regression. Results Suicide severity was associated with multiple visual and auditory markers, including speech prevalence (β=−0.68, P=.02, r2=0.40), overall expressivity (β=−0.46, P=.10, r2=0.27), and head movement measured as head pitch variability (β=−1.24, P=.006, r2=0.48) and head yaw variability (β=−0.54, P=.06, r2=0.32). Conclusions Digital measurements of facial affect, movement, and speech prevalence demonstrated strong effect sizes and linear associations with the severity of suicidal ideation.

Choices of (in)action in obesity: Implications for research on treatment and prevention

“…Finally, addressing active and passive risk-taking patterns is important for developing digital tools to identify individuals struggling with behavioral adherence and who are at greater risk of progressing to overweight and obesity. Utilizing the latest innovation of assessments using digital biomarkers ( 75 ) in combination with machine learning models, could also promote the early detection of active and passive risk patterns. We hope our current perspective will stimulate discussions and further research on active and passive risk-taking in relation to weight control issues, or even expand to other health-related issues.…”

Modern views of machine learning for precision psychiatry

“… 186 Video Facial smile duration, eyebrow movement, disgust expression increased disgust expression in suicidal ideation Galatzer-Levy et al. 189 Eyes gaze angle more non-mutual gazes in MDD Song et al. 190 Gait arm swing and stride reduced arm swing in MDD Shao et al.…”

“…Although body language and facial expressions have always formed a key part of a psychiatric exam, ML has only recently been applied to analyze such data objectively. To date, most work has focused on suicidal ideation, 189 depression, 168 , 169 , 190 schizophrenia, 178 and ASDs. 192 Features derived from overall facial expression, eyes, gait, and posture ( Table 4 ) have been shown to be relevant across many mental disorders.…”

“…As an example, research effort has been dedicated to using computer vision and voice to directly quantify motor activity. Some recent work has demonstrated that digital phenotyping parameters that reflect gross motor activity including speech characteristics (rate of speech, tone) and facial/head movements are associated with suicidal risk, 189 SSRI response in MDD, 251 negative symptomatology in schizophrenia, 191 and Parkinsonian tremor. 252 Such approaches are now being commercialized for all phases of drug development from proof of concept to direct measurement in order to make decisions about ongoing treatment needs.…”

Modern Views of Machine Learning for Precision Psychiatry

“…Although body language and facial expressions form a key part of a psychiatric exam, ML has only recently been applied to analyze such data objectively. To date, most work has been targeted to support the care for suicidal ideation [187] and depression [165], [166], [188], with few studies also dedicated to schizophrenia [175] and autism spectrum disorders [189]. Features derived from video data have been found to be relevant across all of these disorders.…”