Measuring self-regulation in everyday life: Reliability and validity of smartphone-based experiments in alcohol use disorder

Abstract

Self-regulation, the ability to guide behavior according to one’s goals, plays an integral role in understanding loss of control over unwanted behaviors, for example in alcohol use disorder (AUD). Yet, experimental tasks that measure processes underlying self-regulation are not easy to deploy in contexts where such behaviors usually occur, namely outside the laboratory, and in clinical populations such as people with AUD. Moreover, lab-based tasks have been criticized for poor test–retest reliability and lack of construct validity. Smartphones can be used to deploy tasks in the field, but often require shorter versions of tasks, which may further decrease reliability. Here, we show that combining smartphone-based tasks with joint hierarchical modeling of longitudinal data can overcome at least some of these shortcomings. We test four short smartphone-based tasks outside the laboratory in a large sample (N = 488) of participants with AUD. Although task measures indeed have low reliability when data are analyzed traditionally by modeling each session separately, joint modeling of longitudinal data increases reliability to good and oftentimes excellent levels. We next test the measures’ construct validity and show that extracted latent factors are indeed in line with theoretical accounts of cognitive control and decision-making. Finally, we demonstrate that a resulting cognitive control factor relates to a real-life measure of drinking behavior and yields stronger correlations than single measures based on traditional analyses. Our findings demonstrate how short, smartphone-based task measures, when analyzed with joint hierarchical modeling and latent factor analysis, can overcome frequently reported shortcomings of experimental tasks.

Computational phenotyping of aberrant belief updating in individuals with schizotypal traits and schizophrenia

“…First, for reliable intra session measurement, tasks where participants learn about continuous input are preferred over binary inputs; second, simulations can be used to optimize the task generating process (for example, by restricting trajectories); and third, model selection can be contingent on the test-retest reliability of the computational parameters given the data. In line with previous work (Zech et al, 2022), we showed that empirical ICCs are improved with hierarchical estimation, which suggests that more sessions per individual will give more precise ICC estimates, as would standardization of experimental designs and models and extending to response models to include several data sources (e.g. confidence, or physiological data).…”

Intraindividual fluctuations in decision-making predict real-life drinking in alcohol use disorder

“…One explanation for these findings could be that measures related to cognitive control simply do not fluctuate over time. However, an earlier analysis of the tasks revealed both low measurement error and low retest-reliability, implicating that these measures fluctuate substantially even at very short time-scales (Zech, Waltmann, et al, 2022). Other studies that focus on moderators of cognitive control also point toward its short-term malleability (Anderson et al, 2011;Nikolaou et al, 2013;Wilcox et al, 2014).…”

“…Indeed, the prediction of fluctuations in substance use may be crucial to develop earlier treatments for individuals especially those with mild to moderate AUD-i.e., those who are most at risk of deteriorating into more severe forms of AUD (see, for example, Deeken et al, 2022). Task measures may be particularly useful in predicting such changes because they tap into mechanisms putatively underlying substance use (Konova et al, 2020; which likely also fluctuate substantially over time Hedge et al, 2018;Zech, Gable, et al, 2022;Zech, Waltmann, et al, 2022). However, whether such fluctuations in task measures can predict fluctuations in real-life substance use remains unknown and testing this association requires specifically designed longitudinal studies.…”

“…R. Brown et al, 2014;Chetitah et al, 2023) to enhance participant adherence in longitudinal study designs. Crucially, we recently showed that despite their very short duration (~5 min), a set of such gamified tasks (H. R. Brown et al, 2014) represent valid and reliable measurements of cognitive control and decision-making in real-life (Zech, Waltmann, et al, 2022). These tasks can, therefore, be used to study intraindividual relationships between cognitive control, decision-making, and drinking in real-life settings.…”

No evidence for a reciprocal relationship between daily self-control failures and addictive behavior in a longitudinal study

“…(2) An explicit monitoring of SCFs by apps may increase self-reflection and thereby promote the mobilization of cognitive control in response to goal-desire conflicts. In future studies, it will be important to use EMA to assess SCFs and addictive behavior at faster time scales to track fine-grained intraindividual fluctuations of self-control and addictive behaviors and assess associations between the two constructs on a day-to-day basis ( Zech et al, 2022 ).…”