Exploratory study of automated linguistic analysis for progress monitoring and outcome assessment

Abstract

Objective The most successful approach to dealing with treatment failure employs progress monitoring and outcome assessment (PMOA) measures that enable clinicians to identify clients who are failing to progress. On the basis of previous research indicating that word use relates to individuals’ psychological characteristics as well as clinical outcomes, we investigated automated linguistic analysis of client speech as an alternative PMOA approach. Methods We employed the Linguistic Inquiry and Word Count (LIWC) program to study the language of a subset of 12 clients from the York I Depression Study. Automated analyses examined transcripts of 24 individual psychotherapy sessions, one early in treatment (T1) and one late in treatment (T2), split between six good and six poor outcome cases. Results Good outcomes were associated with more positive emotion words and fewer past focus words and negation words at T1. Logistic regression models predicted good versus poor treatment outcome for 70–82% of transcripts. However, analyses failed to support the hypothesis that language use patterns changed from T1 to T2 during the course of therapy. Conclusions Clinicians have been slow to adopt PMOA measures as part of routine practice, partially as a result of additional paperwork and time burdens. Automated analysis of client language provides one alternative method for decreasing client and therapist workload. This study's results, while mixed, provide evidence that additional research should be conducted to investigate the potential for LIWC and other automated measures to provide PMOA data for clinical feedback.

Suitability of Text-Based Communications for the Delivery of Psychological Therapeutic Services to Rural and Remote Communities: Scoping Review

“…Using data from the York I depression study [86], Huston et al [87] examined clients' language use in face-to-face psychotherapy sessions. They analyzed 6 word categories (positive emotion, negative emotion, causation, past focus, negation, and first-person singular pronouns) in 24 sessions, early (T1) and late (T2) for 12 clients identified as having good (n=6) and poor (n=6) treatment outcomes.…”

“…Computational linguistic techniques will continue to evolve as more sophisticated tools are developed to overcome the current limitations. For example, LIWC is unable to detect or compute context and cannot account for changes in meaning resulting from irony, sarcasm, or idioms [93], nor does it take into account negations [87] or qualifiers before words. Predefined LIWC dictionaries may not be sufficiently broad to account for the categories of interest [93].…”

Modeling Linguistic (A)Synchrony: A Case Study of Therapist–Client Interaction

“…Recent studies ( Huston et al, 2019 ; Tay, 2020 ; Qiu and Tay, 2022 ) highlight the potential for the four summary variables to profile how language is used in therapeutic work. They reflect aspects like how narratives are told, the stance of therapists when dispensing advice and of clients when receiving it, the negotiation of relationships, and linguistic displays of emotional states.…”

A voice without a mouth no more: The neurobiology of language and consciousness

“…Supporting the proposal suggested here, a growing body of research shows that the words people use are predictive of college grades, life expectancy, personality traits, mood, wellbeing, mental health, and recovery from mental health problems (Al-Mosaiwi and Johnstone, 2018;D'Andrea et al, 2012;Fast and Funder, 2008;Luhmann, 2017;Pennebaker et al, 2014Pennebaker et al, , 2003Penzel et al, 2017;Pressman and Cohen, 2007;Robinson et al, 2013;Rude et al, 2004). For example, more first-person singular pronouns (i.e., 'I-talk'), more negative and less positive emotion words and fewer cognitive process words distinguish people with depression and other mental health problems and changes in the use of these words predict recovery (Edwards and Holtzman, 2017;Huston et al, 2019;Pennebaker et al, 2003;Tackman et al, 2019;Tølbøll, 2019;Zimmermann et al, 2017). Self-talk (Kross et al, 2014;Reichl et al, 2013;Treadwell and Kendall, 1996) and aspects of inner speech (particularly its dialogic nature) predict a wide variety of mental health outcomes (Alderson-Day et al, 2018;de Sousa et al, 2016;Luo and McAloon, 2021;Rosen et al, 2021Rosen et al, , 2020Rosen et al, , 2018Wallace et al, 2009;Whitehouse et al, 2006).…”