Computational psychotherapy research: Scaling up the evaluation of patient–provider interactions.

Abstract

In psychotherapy, the patient-provider interaction contains the treatment’s active ingredients. However, the technology for analyzing the content of this interaction has not fundamentally changed in decades, limiting both the scale and specificity of psychotherapy research. New methods are required in order to “scale up” to larger evaluation tasks and “drill down” into the raw linguistic data of patient-therapist interactions. In the current paper we demonstrate the utility of statistical text analysis models called topic models for discovering the underlying linguistic structure in psychotherapy. Topic models identify semantic themes (or topics) in a collection of documents (here, transcripts). We used topic models to summarize and visualize 1,553 psychotherapy and drug therapy (i.e., medication management) transcripts. Results showed that topic models identified clinically relevant content, including affective, content, and intervention related topics. In addition, topic models learned to identify specific types of therapist statements associated with treatment related codes (e.g., different treatment approaches, patient-therapist discussions about the therapeutic relationship). Visualizations of semantic similarity across sessions indicate that topic models identify content that discriminates between broad classes of therapy (e.g., cognitive behavioral therapy vs. psychodynamic therapy). Finally, predictive modeling demonstrated that topic model derived features can classify therapy type with a high degree of accuracy. Computational psychotherapy research has the potential to scale up the study of psychotherapy to thousands of sessions at a time, and we conclude by discussing the implications of computational methods such as topic models for the future of psychotherapy research and practice.

A Comparison of Natural Language Processing Methods for Automated Coding of Motivational Interviewing

“…The two methods, introduced and described below, differ substantially in terms of their complexity and their conceptualization of language and linguistic structure. Moreover, they are fundamentally different models relative to topic models or maximum entropy models tested in previous research (Atkins et al, 2014; Can et al, in press; Imel et al, 2015). The most important difference between the models tested in this study and topic models is that these models attempt to incorporate the linguistic structure of a sentence beyond the mere presence of words and phrases.…”

“…The methods tested here differ from other research predictive models based on topic modeling (also called Latent Dirichlet Allocation; e.g., Atkins et al, 2014; Imel et al, 2015; Schwartz et al, 2013) and word-count methods using predefined keywords (e.g., Linguistic Inquiry and Word Count [LIWC]; Tausczik & Pennebaker, 2010) in a few key ways. First, unlike topic models, the present methods do not identify latent “topics” based on patterns of word co-occurrences.…”

“…Several limitations may have contributed to the model’s difficulty with some codes; for example, topic models use a “bag of words” approach, where the functional relationships and ordering between words are ignored. Imel, Steyvers, and Atkins (2015) used similar topic modeling techniques with a larger corpus of 1,553 psychotherapy sessions (including, but not limited to MI and substance abuse treatment) and found topic models were able to identify various treatment-related topics (e.g., emotions, relationships) and were able to discriminate different types of treatments (e.g., psychodynamic, humanistic, and medication management therapies).…”

Emotional Expression Predicts Treatment Outcome in Focal Psychodynamic and Cognitive Behavioural Therapy for Anorexia Nervosa: Findings from the ANTOP Study

“…However, it bears advantages over self-ratings and observer ratings such as high objectivity. Moreover, it is considered a method that could significantly contribute to a better understanding of the active ingredients of psychotherapy [12]. Future studies may also apply more sophisticated text-mining approaches (e.g., n -grams, topic models) that include machine-learning procedures [12].…”

"It's hard to argue with a computer"

“…Gold-standard training and supervision involves recording sessions, deriving performance-based feedback from session recordings, and providing expert consultation and coaching. Unfortunately, this approach is time consuming, expensive, and does not scale up to real-world settings [1, 23]. Due to time, resources, and confidentiality issues, in many clinics supervision either does not occur or is often based upon the self-report of the counselor rather than independent observation of clinical sessions [17].…”