An Unsupervised Learning Approach for Detecting Relapses from Spontaneous Speech in Patients with Psychosis

Garoufis, Christos; Zlatintsi, Athanasia; Filntisis, Panagiotis P.; Efthymiou, Niki; Kalisperakis, Emmanouil; Karantinos, Thomas; Mantonakis, Leonidas; Smyrnis, Nikolaos; Maragos, Petros

doi:10.1109/bhi50953.2021.9508515

Cited by 7 publications

(6 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We opted for an unsupervised learning approach since it provides the advantage that models can be trained without necessarily having access to data from relapsing periods. Experiments conducted in a database with a total of 16 patients, containing over 38,000 s of total speech, yielded encouraging results for both classical Convolutional Autoencoders (CAEs) and Convolutional Variational Autoencoders (CVAEs) in a personalized setting, in agreement to our previous results derived from smaller subsets of this database [ 86 , 119 ]. Moreover, CVAEs can reach the performance of personalized models in a global (patient-independent) setup, especially when per-person normalization is applied to the input features [ 119 ].…”

Section: Material Methods and Research Resultssupporting

confidence: 85%

“…This distribution is encouraged to align with the spherical isotropic Gaussian,

, through the imposition of a Kullback–Liebler divergence loss (

) in the bottleneck of the network. For more details about the developed architectures, we refer the reader to [ 86 , 119 ].…”

Section: Material Methods and Research Resultsmentioning

confidence: 99%

“…This is especially significant in mental health monitoring, where the availability of data corresponding to relapsing states is scarce. Such unsupervised anomaly detection algorithms, such as, for instance, autoencoder neural networks, have been developed and applied on audio signals [ 86 , 87 ] and medical images [ 88 ], as well as data collected from various passive sensors [ 48 , 89 , 90 , 91 ].…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

E-Prevention: Advanced Support System for Monitoring and Relapse Prevention in Patients with Psychotic Disorders Analyzing Long-Term Multimodal Data from Wearables and Video Captures

Zlatintsi

Filntisis

Garoufis

et al. 2022

Sensors

View full text Add to dashboard Cite

Wearable technologies and digital phenotyping foster unique opportunities for designing novel intelligent electronic services that can address various well-being issues in patients with mental disorders (i.e., schizophrenia and bipolar disorder), thus having the potential to revolutionize psychiatry and its clinical practice. In this paper, we present e-Prevention, an innovative integrated system for medical support that facilitates effective monitoring and relapse prevention in patients with mental disorders. The technologies offered through e-Prevention include: (i) long-term continuous recording of biometric and behavioral indices through a smartwatch; (ii) video recordings of patients while being interviewed by a clinician, using a tablet; (iii) automatic and systematic storage of these data in a dedicated Cloud server and; (iv) the ability of relapse detection and prediction. This paper focuses on the description of the e-Prevention system and the methodologies developed for the identification of feature representations that correlate with and can predict psychopathology and relapses in patients with mental disorders. Specifically, we tackle the problem of relapse detection and prediction using Machine and Deep Learning techniques on all collected data. The results are promising, indicating that such predictions could be made and leading eventually to the prediction of psychopathology and the prevention of relapses.

show abstract

Section: Material Methods and Research Resultssupporting

confidence: 85%

“…This distribution is encouraged to align with the spherical isotropic Gaussian,

, through the imposition of a Kullback–Liebler divergence loss (

) in the bottleneck of the network. For more details about the developed architectures, we refer the reader to [ 86 , 119 ].…”

Section: Material Methods and Research Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

E-Prevention: Advanced Support System for Monitoring and Relapse Prevention in Patients with Psychotic Disorders Analyzing Long-Term Multimodal Data from Wearables and Video Captures

Zlatintsi

Filntisis

Garoufis

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…The researchers first created spectograms from audio signals and then extracted features of them. Garoufis et al ( 59 , 60 ) utilized unsupervised learning with Convolutional Variational Autoencoders (CVAE), a type of generative model, to learn latent representations of speech data. By comparing the reconstructed speech to the original, the model identified deviations that may indicate relapse episodes.…”

Section: State-of-the Art Approaches To the Automated Analysis Of Spe...mentioning

confidence: 99%

Automated analysis of speech as a marker of sub-clinical psychotic experiences

Olah,

Spencer,

Cummins

et al. 2024

Front. Psychiatry

View full text Add to dashboard Cite

Automated speech analysis techniques, when combined with artificial intelligence and machine learning, show potential in capturing and predicting a wide range of psychosis symptoms, garnering attention from researchers. These techniques hold promise in predicting the transition to clinical psychosis from at-risk states, as well as relapse or treatment response in individuals with clinical-level psychosis. However, challenges in scientific validation hinder the translation of these techniques into practical applications. Although sub-clinical research could aid to tackle most of these challenges, there have been only few studies conducted in speech and psychosis research in non-clinical populations. This work aims to facilitate this work by summarizing automated speech analytical concepts and the intersection of this field with psychosis research. We review psychosis continuum and sub-clinical psychotic experiences, and the benefits of researching them. Then, we discuss the connection between speech and psychotic symptoms. Thirdly, we overview current and state-of-the art approaches to the automated analysis of speech both in terms of language use (text-based analysis) and vocal features (audio-based analysis). Then, we review techniques applied in subclinical population and findings in these samples. Finally, we discuss research challenges in the field, recommend future research endeavors and outline how research in subclinical populations can tackle the listed challenges.

show abstract

“…Hanai et al developed diarization methods for recorded neuropsychological exams [12], which were used by Lin et al in a longitudinal study to identify digital voice biomarkers for cognitive health [13]. Garoufis et al used diarization methods as a part of speech analysis algorithms to detect relapses in patients with psychotic disorders such as bipolar disorder and schizophrenia [14]. Hansen et al investigated the role of diarization in developing generalized speechbased emotion recognition models, particularly with the goal of identifying depression and remission of the condition [15].…”

Section: Introductionmentioning

confidence: 99%

An Effective Automated Algorithm to Isolate Patient Speech from Conversations with Clinicians

Jaquenoud

Keene

Shlayan

et al. 2022

Preprint

View full text Add to dashboard Cite

A growing number of algorithms are being developed to automatically identify disorders or disease biomarkers from digitally recorded audio of patient speech. An important step in these analyses is to identify and isolate the patient's speech from that of other speakers or noise that are captured in a recording. However, current algorithms, such as diarization, only label the identified speech segments in terms of non-specific speakers, and do not identify the specific speaker of each segment, e.g., clinician and patient. In this paper, we present a novel algorithm that not only performs diarization on clinical audio, but also identifies the patient among the speakers in the recording and returns an audio file containing only the patient's speech. Our algorithm first uses pretrained diarization algorithms to separate the input audio into different tracks according to non-specific speaker labels. Next, in a novel step not conducted in other diarization tools, the algorithm uses the average loudness (quantified as power) of each audio track to identify the patient, and return the audio track containing only their speech. Using a practical expert-based evaluation methodology and a large dataset of clinical audio recordings, we found that the best implementation of our algorithm achieved near-perfect accuracy on two validation sets. Thus, our algorithm can be used for effectively identifying and isolating patient speech, which can be used in downstream expert and/or data-driven analyses.

show abstract

An Unsupervised Learning Approach for Detecting Relapses from Spontaneous Speech in Patients with Psychosis

Cited by 7 publications

References 23 publications

E-Prevention: Advanced Support System for Monitoring and Relapse Prevention in Patients with Psychotic Disorders Analyzing Long-Term Multimodal Data from Wearables and Video Captures

E-Prevention: Advanced Support System for Monitoring and Relapse Prevention in Patients with Psychotic Disorders Analyzing Long-Term Multimodal Data from Wearables and Video Captures

Automated analysis of speech as a marker of sub-clinical psychotic experiences

An Effective Automated Algorithm to Isolate Patient Speech from Conversations with Clinicians

Contact Info

Product

Resources

About