Using machine learning of computerized vocal expression to measure blunted vocal affect and alogia

Abstract

Negative symptoms are a transdiagnostic feature of serious mental illness (SMI) that can be potentially “digitally phenotyped” using objective vocal analysis. In prior studies, vocal measures show low convergence with clinical ratings, potentially because analysis has used small, constrained acoustic feature sets. We sought to evaluate (1) whether clinically rated blunted vocal affect (BvA)/alogia could be accurately modelled using machine learning (ML) with a large feature set from two separate tasks (i.e., a 20-s “picture” and a 60-s “free-recall” task), (2) whether “Predicted” BvA/alogia (computed from the ML model) are associated with demographics, diagnosis, psychiatric symptoms, and cognitive/social functioning, and (3) which key vocal features are central to BvA/Alogia ratings. Accuracy was high (>90%) and was improved when computed separately by speaking task. ML scores were associated with poor cognitive performance and social functioning and were higher in patients with schizophrenia versus depression or mania diagnoses. However, the features identified as most predictive of BvA/Alogia were generally not considered critical to their operational definitions. Implications for validating and implementing digital phenotyping to reduce SMI burden are discussed.

High Predictive Accuracy of Negative Schizotypy With Acoustic Measures

“…Although self-report ecological momentary assessment solutions for measuring state negative schizotypy have been advanced (Brown et al, 2007), acoustic analyses (and other digital phenotyping technologies) offer potentially greater precision for measuring potential changes in signal because they employ active/passive recording technologies, yield ratio-level data, and are amenable to multimodal and high-dimensional data analytic approaches. The present findings are similar to those found in a study of patients with serious mental illness that used acoustic features to predict negative symptom clinical ratings (Cohen, Cox, et al, 2020). In that study, we achieved similar accuracy rates as in the present study (generally, > 80%; in some cases, > 90%), which suggests that acoustic-based digital phenotyping is appropriate for measuring negative traits/symptoms across the schizophrenia spectrum.…”

“…This feature selection approach assumes continuity in phenotypic expression between negative symptoms and negative schizotypy. Although implied in some schizophrenia-spectrum models (Lenzenweger, 2006; Meehl, 1962), it is not clear that this continuity actually exists, and it is not clear that these conceptually critical features are particularly important even in schizophrenia (Cohen, Auster, McGovern, & MacAulay, 2014; Cohen, Cox, et al, 2020; Cohen, Mitchell, et al, 2016; Meaux et al, 2018). Moreover, the reliance on small feature sets (typically on the order of two to 10 features) is a problem in that the human voice is complex and can be quantified in thousands of potentially distinct summary features.…”

Avolition as the core negative symptom in schizophrenia: relevance to pharmacological treatment development

“…To further improve the assessment of negative symptoms and obtain longitudinal rather than cross-sectional data, a third-generation of measurement tools, based on digital phenotyping (i.e., the use of mobile devices, such as smartphones and smartbands, to initiate data collection in everyday life) is currently under development. Both active (e.g., ecological momentary assessment surveys, ambulatory videos) and passive (e.g., geolocation, accelerometry, acoustic measures) digital phenotyping measures may hold promise for measuring negative symptoms more objectively in the context of everyday life [39][40][41][42] .…”

Acceptability of collecting speech samples from the elderly via the telephone

“…Speech samples have been analyzed with acoustic feature extraction software (e.g., openSMILE 46 ) and studies have shown that these features can have high predictive potential in patients with serious mental illness. 47 , 48 Furthermore, word timing features can be automatically extracted to analyze speed and rhythm in speech. In sum, while fairly high word error rates may exist in automated transcriptions, other modalities of features which are not affected by these errors can still be accurately extracted and used in automated systems.…”