Motion energy analysis during speech tasks in medication-naïve individuals with at-risk mental states for psychosis

Abstract

Movement abnormalities are commonly observed in schizophrenia and at-risk mental states (ARMS) for psychosis. They are usually detected with clinical interviews, such that automated analysis would enhance assessment. Our aim was to use motion energy analysis (MEA) to assess movement during free-speech videos in ARMS and control individuals, and to investigate associations between movement metrics and negative and positive symptoms. Thirty-two medication-naïve ARMS and forty-six healthy control individuals were filmed during speech tasks. Footages were analyzed using MEA software, which assesses movement by differences in pixels frame-by-frame. Two regions of interest were defined—head and torso—and mean amplitude, frequency, and coefficient of variability of movements for them were obtained. These metrics were correlated with the Structured Interview for Prodromal Syndromes (SIPS) symptoms, and with the risk of conversion to psychosis—inferred with the SIPS risk calculator. ARMS individuals had significantly lower mean amplitude of head movement and higher coefficients of movement variability for both head and torso, compared to controls. Higher coefficient of variability was related to higher risk of conversion. Negative correlations were seen between frequency of movement and most SIPS negative symptoms. All positive symptoms were correlated with at least one movement variable. Movement abnormalities could be automatically detected in medication-naïve ARMS subjects by means of a motion energy analysis software. Significant associations of movement metrics with symptoms were found, supporting the importance of movement analysis in ARMS. This could be a potentially important tool for early diagnosis, intervention, and outcome prediction.

Deictic gestures impairment in individuals with at risk mental states for psychosis.

“…At last, we observed that gesture impairments were related to differences in energy of movement-assessed in a previous work of ours. In these ndings, a reduced movement and increased variability of movements in ARMS was observed with MEA 20 . Given that MEA only measures how much the person moved and not the type of movement made, in the present study we were able to see that gesticulation was positively correlated with amount of movement and negatively correlated with movement variability.…”

“…MEA detects movement using a frame-differencing method that evaluates differences in grayscale pixels frame by frame and quanti es it as energy motion 18,36 . More information about the collection of this data and the results obtained can be found in our article 20 .…”

“…We have published an analysis similar to one performed by Dean et al (2018) 19 to assess the amount of general movement (head and torso) in 32 medication-naïve ARMS individuals compared to forty-six healthy controls during speech tasks (see more information in Lopes-Rocha et al, 2022). A reduced total movement frequency and increased movement variability was seen for ARMS subjects-which means that they move less, but when they move they do it in a more erratic way 20 . Furthermore, energy of movement variables were signi cantly correlated with positive and negative symptoms.…”

Gesticulation in individuals with at risk mental states for psychosis

“…We have published an analysis similar to one performed by Dean et al (2018) 20 to assess the amount of general movement in ARMS during speech tasks (see more information in Lopes-Rocha et al, 2022). A reduced total movement frequency and increased movement variability was seen for ARMS subjects, and these variables were significantly correlated with positive and negative symptoms 21 . However, it was not possible to infer if MEA findings were due to gesture deficits or not, as the software only “sees” general movement and is not capable of discriminating gestures.…”

Ethical Implications of the Use of Language Analysis Technologies for the Diagnosis and Prediction of Psychiatric Disorders

“…For individuals at first-episode psychosis, AUCs of up to 0.92 were obtained to predict who would receive the diagnosis of schizophrenia 6 months in advance [ 16 ]. Audio features such as pauses [ 18 ] and nonverbal behavior such as gestures [ 19 ] and movement in general [ 20 ] were also seen to be discriminative between healthy and ARMSs. Besides serious mental disorders [ 21 ], AI frameworks have also been developed to detect and classify other mental disorders, as shown by numerous publications and challenges to establish an accurate depression estimation from audio, video, and text information [ 22 , 23 ].…”