Aggregating and Analyzing Daily Drinking Data in Clinical Trials: A Comparison of Type I Errors, Power, and Bias

Abstract

ABSTRACT. Objective: Statistical analyses in alcohol clinical trials often use longitudinal daily drinking data (e.g., percentage of drinking days) to test treatment efficacy. Such data can be aggregated and analyzed in many ways. To assess how statistical analytic decisions may influence substantive results, the current report compares different aggregation methods (e.g., computing percentages of drinking days vs. using daily binary indicators of drinking) and statistical methods (i.e., least squares regression, linear mixed-effects models [LMM], generalized linear mixed models [GLMM], and generalized estimating equations [GEE]) for testing the effects of treatment on drinking outcomes in clinical trials. Method: A simulation study repeatedly resampled daily drinking data from the treatment period of the Combined Pharmacotherapies and Behavioral Interventions for Alcohol Dependence (COMBINE) Study at different sample sizes. Treatment effects in each data set were modeled using different aggregation and statistical methods. Results: Type I error rates were near the expected rate for most models but on occasion were mildly elevated when disaggregated daily drinking data were analyzed using GLMM or GEE with an exchangeable correlation structure. Most methods yielded similar statistical power, although power decreased when modeling disaggregated daily drinking with GLMM and had mixed increases and decreases when the longitudinal nature of data was ignored by using fully aggregated data with independent samples t tests. Conclusions: When testing treatment main effects, relatively simpler statistical methods with fewer repeated measures may perform equally well or better than more complicated methods. Patterns of significance and treatment effect size estimates are likely comparable across most studies that use different aggregation and statistical methods, but differences between these methods may occasionally have an important impact on conclusions in clinical trials. (J. Stud. Alcohol Drugs, 77, 986-991, 2016)

A randomized trial of female-specific cognitive behavior therapy for alcohol dependent women.

“…Growth curve models assumed Gaussian outcomes despite the non-normal drinking variable distributions. Other approaches were considered that make fewer distributional assumptions (e.g., generalized estimating equations, generalized linear mixed models) but these have been shown to provide similar effect estimates and significance tests for testing treatment outcomes in growth curve models as multilevel models that assume normal distributions (Hallgren, Atkins, & Witkiewitz, 2016). Gaussian models were chosen instead of multilevel zero-inflated models, to reduce complexity and enhance interpretability.…”

The rise and fall of substance use during treatment: Applying recency and expectancy principles to daily substance use patterns.

“…Substance use outcomes in clinical trials are often analyzed as aggregated percentages or counts of substance use over time (e.g., percent days abstinent, percent heavy drinking days, maximum days of continuous abstinence; Hallgren et al, 2016; McCann et al, 2015). However, substance use motivation is not static between days or even within a day, but rather covaries temporally with the presence of endogenous and exogenous stimuli associated with substance use (Benitez & Goldman, 2019; Epstein & Preston, 2010; Metrik et al, 2018; Monk & Heim, 2014; Palij et al, 1996; Rhew et al, 2021; Serre et al, 2015).…”

“…Analyzing aggregated daily substance use data necessarily conflates temporal patterns, resulting in the loss of potentially valuable information about temporally dynamic changes in substance use motivation. Disaggregated daily substance use patterns during treatment may reveal with greater precision when substance use motivation is surging, even if statistical methods for analyzing treatment endpoints do not yield substantively different conclusions between aggregated and disaggregated data (Hallgren et al, 2016). More precise temporal patterns may inform both addiction theory and clinical practice by identifying how and when substance use is most likely to occur, which could reveal how treatment mechanisms operate and optimize the timing of treatment delivery (Nahum-Shani et al, 2018).…”