Time to get personal? The impact of researchers choices on the selection of treatment targets using the experience sampling methodology

Abstract

One of the promises of the experience sampling methodology (ESM) is that a statistical analysis of an individual's emotions, cognitions and behaviors in everyday-life could be used to identify relevant treatment targets. A requisite for clinical implementation is that outcomes of such person-specific time-series analyses are not wholly contingent on the researcher performing them. Methods To evaluate this, we crowdsourced the analysis of one individual patient's ESM data to 12 prominent research teams, asking them what symptom(s) they would advise the treating clinician to target in subsequent treatment. Results Variation was evident at different stages of the analysis, from preprocessing steps (e.g., variable selection, clustering, handling of missing data) to the type of statistics and rationale for selecting targets. Most teams did include a type of vector autoregressive model, examining relations between symptoms over time. Although most teams were confident their selected targets would provide useful information to the clinician, not one recommendation was similar: both the number (0-16) and nature of selected targets varied widely. Conclusion This study makes transparent that the selection of treatment targets based on personalized models using ESM data is currently highly conditional on subjective analytical choices and highlights key conceptual and methodological issues that need to be addressed in moving towards clinical implementation.

A Review of Explicit and Implicit Assumptions When Providing Personalized Feedback Based on EMA Data.

“…In many designs, people are asked about their experiences in the exact moment (e.g., "how sad are you feeling right now? "; c.f., Thomas & Diener, 1990;Parkinson, Briner, Reynolds, & Totterdell, 1995;Barrett, 1997;Ebner-Priemer et al, 2006;Ben-Zeev, Young, & Madsen, 2009;Ben-Zeev & Young, 2010;Dockray et al, 2010;Bylsma, Clift, & Rottenberg, 2011;Ben-Zeev, McHugo, Xie, Dobbins, & Young, 2012;Wenze, Gunthert, & German, 2012;Kramer et al, 2014;van der Krieke et al, 2016;Kroeze et al, 2017;Lay, Gerstorf, Scott, Pauly & Hoppmann, 2017;Colombo et al, 2019;2020;Neubauer, Scott, Sliwinski, & Smyth, 2019;Lucas et al, 2020;Ornee, Bastiaansen, & Oldehinkel, 2020), which as mentioned earlier, seems closely related to the ES. An important drawback to this procedure however, is that there is a high chance that salient experiences will be missed.…”

“…For affective measures, it is generally reported that RA scores are higher than the mean that is observed in EMA data (Thomas & Diener, 1990 6 ;Parkinson et al 1995 7 ;Ebner-Priemer et al, 2006;Ben-Zeev & Young, 2009;Wenze et al, 2012;Lay et al, 2017;Colombo et al, 2019;2020 8 ), although not in all studies (Barrett, 1997). This difference between RA and aggregated aggregated EMA scores for participants suffering from depression symptoms (Colombo et al, 2019;2020), or borderline personality disorder (Ebner-Priemer et al, 2006). Some authors report that healthy participants showed specifically higher RA-scores than aggregated EMAscores for positive affect compared to depressed or anxious participants (Ben-Zeev et al, 2009;Wenze et al, 2012;Colombo et al, 2019;2020).…”

“…This difference between RA and aggregated aggregated EMA scores for participants suffering from depression symptoms (Colombo et al, 2019;2020), or borderline personality disorder (Ebner-Priemer et al, 2006). Some authors report that healthy participants showed specifically higher RA-scores than aggregated EMAscores for positive affect compared to depressed or anxious participants (Ben-Zeev et al, 2009;Wenze et al, 2012;Colombo et al, 2019;2020).…”

Latent Markov Latent Trait Analysis for Exploring Measurement Model Changes in Intensive Longitudinal Data

“…Finally, we would like to highlight that there is no gold standard yet in how to analyze intensive longitudinal data and the latent variable framework that LMLTA is based on is only one possibility. There are various other reasonable frameworks for analyzing the data (e.g., network psychometrics; Epskamp, 2020;Marsman et al, 2018) and decisions about the data analysis can considerably impact, for example, clinical recommendations (Bastiaansen et al, 2020). Therefore, in the future, it would be desirable to compare perspectives about psychological phenomena from various modeling approaches.…”

ESMvis: a tool for visualizing individual Experience Sampling Method (ESM) data

“…Although increasing the complexity of ESM-based feedback may hold great promise for getting the most out of ESM data, the usefulness of this approach for patients is still unclear. The interpretation of VAR-based network models is highly depended on the exact network model used and different choices regarding a VAR based network model can lead to widely different recommendations for the patient [31,32]. This not only results in conflicting advice on where and how to perform further interventions, but also in the additional complication that at the moment we do not have well developed tools to evaluate these different outcomes.…”