Compliance trends in a 14-week ecological momentary assessment study of undergraduate alcohol drinkers

Abstract

High compliance is a priority for successful ecological momentary assessment (EMA) research, but meta-analyses of between-study differences show that reasons for missed prompts remain unclear. We examined compliance data from a 14-week, 182-survey EMA study of undergraduate alcohol use to test differences over time and across survey types between participants with better and worse compliance rates, and to evaluate the impact of incentives on ongoing participation. Participants were n=196 students (65.8% Female; Mage=20.6). Overall compliance was 76.4%, declining gradually from 88.9% to 70% over 14 weeks. Declines were faster in participants with lower overall compliance, but we found no demographic, personality, mental health, or substance use differences between participants with better versus worse compliance rates. Compliance varied by survey type, and unannounced bonus incentives did not impact compliance rates. Participants completed fewer surveys the week after winning a gift card. We offer recommendations for designing future EMA studies.

Respondent characteristics associated with adherence in a general population ecological momentary assessment study

“…On the other hand, some alcohol use is quite normative among young adults (Willoughby et al., 2021) and abstaining from or low alcohol use could represent a marker of, for example, poor health or social functioning. Previous studies have suggested that general population variation in alcohol use (not distinguishing beer/alcopops vs. spirits) may not, in fact, be associated with EMA adherence (Howard & Lamb, 2022). In contrast, participants with a substance use disorder have been shown to have lower average EMA adherence (Jones et al., 2019).…”

A Personalized Data Dashboard to Improve Compliance with Ecological Momentary Assessments: Protocol for a Study to Compare EMA as Usual to EMA with a Data Dashboard Deployed in a Micro-Randomized Trial (Preprint)

“…However, compliance -particularly in substance use studies -can be a major concern for several reasons. First, young adults are less likely to complete reports on weekends, when substance use is more likely to occur (e.g., Howard & Lamb [5]; Sokolovsky et al [6]) and thus most critical to assess. Second, particular types of substance use behaviors -such as combining alcohol and cannabisoccurs somewhat rarely (e.g., Linden-Carmichael et al [7]).…”

Recruitment and retention of adolescents for an ecological momentary assessment measurement burst mental health study: The MHIM engagement strategy

“…One recent study also highlighted the possibility that some strategies can have a negative effect, for example, in a student alcohol use study, compliance declined immediately following the award of bonuses. 54 There is little evidence regarding the effects of leveraging incentives in biosampling research. In one qualitative interview study exploring the perspectives of underrepresented participants concerning their involvement in biospecimen research, approximately half of the participants reported monetary incentives as a motivating factor for providing hair samples.…”

“…It is also important to consider that some strategies could have a paradoxical counterproductive effect. One recent study also highlighted the possibility that some strategies can have a negative effect, for example, in a student alcohol use study, compliance declined immediately following the award of bonuses 54 . There is little evidence regarding the effects of leveraging incentives in biosampling research.…”