Online interventions hold great potential for Therapeutic Change Process Research (TCPR), a field that aims to relate in-therapeutic change processes to the outcomes of interventions. Online a client is treated essentially through the language their counsellor uses, therefore the verbal interaction contains many important ingredients that bring about change. TCPR faces two challenges: how to derive meaningful change processes from texts, and secondly, how to assess these complex, varied, and multi-layered processes? We advocate the use text mining and multi-level models (MLMs): the former offers tools and methods to discovers patterns in texts; the latter can analyse these change processes as outcomes that vary at multiple levels. We (re-)used the data from Lamers et al. ( 2015 ) because it includes outcomes and the complete online intervention for clients with mild depressive symptoms. We used text mining to obtain basic text-variables from e-mails, that we analyzed through MLMs. We found that we could relate outcomes of interventions to variables containing text-information. We conclude that we can indeed bridge text mining and MLMs for TCPR as it was possible to relate text-information (obtained through text mining) to multi-leveled TCPR outcomes (using a MLM). Text mining can be helpful to obtain change processes, which is also the main challenge for TCPR. We showed how MLMs and text mining can be combined, but our proposition leaves open how to obtain the most relevant textual operationalization of TCPR concepts. That requires interdisciplinary collaboration and discussion. The future does look bright: based on our proof-of-concept study we conclude that MLMs and text mining can indeed advance TCPR.

Therapeutic Change Process Research (TCPR) connects within-therapeutic change processes to outcomes. The labour intensity of qualitative methods limit their use to small scale studies. Automated text-analyses (e.g. text mining) provide means for analysing large scale text patterns. We aimed to provide an overview of the frequently used qualitative text-based TCPR methods and assess the extent to which these methods are reliable and valid, and have potential for automation. We systematically reviewed PsycINFO, Scopus, and Web of Science to identify articles concerning change processes and text or language. We evaluated the reliability and validity based on replicability, the availability of code books, training data and inter-rater reliability, and evaluated the potential for automation based on the example- and rule-based approach. From 318 articles we identified four often used methods: Innovative Moments Coding Scheme, the Narrative Process Coding Scheme, Assimilation of Problematic Experiences Scale, and Conversation Analysis. The reliability and validity of the first three is sufficient to hold promise for automation. While some text features (content, grammar) lend themselves for automation through a rule-based approach, it should be possible to automate higher order constructs (e.g. schemas) when sufficient annotated data for an example-based approach are available.

The linear mixed effects model is an often used tool for the analysis of multilevel data. However, this model has an ill-understood shortcoming: it assumes that observations within clusters are always positively correlated. This assumption is not always true: individuals competing in a cluster for scarce resources are negatively correlated. Random effects in a mixed effects model can model a positive correlation among clustered observations but not a negative correlation. As negative clustering effects are largely unknown to the sheer majority of the research community, we conducted a simulation study to detail the bias that occurs when analysing negative clustering effects with the linear mixed effects model. We also demonstrate that ignoring a small negative correlation leads to deflated Type-I errors, invalid standard errors and confidence intervals in regression analysis. When negative clustering effects are ignored, mixed effects models incorrectly assume that observations are independently distributed. We highlight the importance of understanding these phenomena through analysis of the data from Lamers, Bohlmeijer, Korte, and Westerhof (2015). We conclude with a reflection on well-known multilevel modelling rules when dealing with negative dependencies in a cluster: negative clustering effects can, do and will occur and these effects cannot be ignored.

Nowadays, traditional forms of psychotherapy are increasingly complemented by online interactions between client and counselor. In (some) web-based psychotherapeutic interventions, meetings are exclusively online through asynchronous messages. As the active ingredients of therapy are included in the exchange of several emails, this verbal exchange contains a wealth of information about the psychotherapeutic change process. Unfortunately, drop-out-related issues are exacerbated online. We employed several machine learning models to find (early) signs of drop-out in the email data from the “Alcohol de Baas” intervention by Tactus. Our analyses indicate that the email texts contain information about drop-out, but as drop-out is a multidimensional construct, it remains a complex task to accurately predict who will drop out. Nevertheless, by taking this approach, we present insight into the possibilities of working with email data and present some preliminary findings (which stress the importance of a good working alliance between client and counselor, distinguish between formal and informal language, and highlight the importance of Tactus' internet forum).

This thesis is part of the What Works When for Whom project, supported by the Life Science & eHealth domain of the Accelerating Scientific Discovery (ASDI) call from the Netherlands eScience Center (NLeSC; Amsterdam, the Netherlands): grant number 027.015.G04 awarded to dr. A. M. Sools. The NLeSC is the national knowledge center for the development and application of research software to advance scientific research, and is funded by the Netherlands Organization for Scientific Research (in Dutch: Nederlandse organisatie voor Wetenschappelijk Onderzoek) and SURF (Samenwerkende Universitaire Rekenfaciliteiten). The project received no additional funding, which allowed us to conduct all of the thesis' research projects in the absence of any conflicting interests. Printing was financially supported by the University of Twente. DesignRob Smink ("na een volledige lezing zullen alle tekeningen duidelijk zijn")

The multilevel model (MLM) is the popular approach to describe dependences of hierarchically clustered observations. A main feature is the capability to estimate (cluster-specific) random effect parameters, while their distribution describes the variation across clusters. However, the MLM can only model positive associations among clustered observations, and it is not suitable for small sample sizes. The limitation of the MLM becomes apparent when estimation methods produce negative estimates for random effect variances, which can be seen as an indication that observations are negatively correlated. A gentle introduction to Bayesian covariance structure modeling (BCSM) is given, which makes it possible to model also negatively correlated observations. The BCSM does not model dependences through random (cluster-specific) effects, but through a covariance matrix. We show that this makes the BCSM particularly useful for small data samples. We draw specific attention to detect effects of a personalized intervention. The effect of a personalized treatment can differ across individuals, and this can lead to negative associations among measurements of individuals who are treated by the same therapist. It is shown that the BCSM enables the modeling of negative associations among clustered measurements and aids in the interpretation of negative clustering effects. Through a simulation study and by analysis of a real data example, we discuss the suitability of the BCSM for small data sets and for exploring effects of individualized treatments, specifically when (standard) MLM software produces negative or zero variance estimates.