Compliance with ecological momentary assessment protocols in substance users: a meta‐analysis

Abstract

Background and Aims While there are considerable benefits to Ecological Momentary Assessment (EMA), poor compliance with assessment protocols has been identified as a limitation, particularly in substance users. Our aim was to identify the pooled compliance rate of EMA studies in substance users and examine variables that may influence compliance with EMA protocols, such as the length and frequency of assessments. Design A meta‐analysis and meta‐regression of all possible studies (randomized controlled trials and longitudinal) which incorporated EMA protocols, examining substance use. Setting Studies took place from 1998 to 2017, in numerous countries world‐wide. Participants One hundred and twenty‐six studies were identified, contributing a total of 19 431 participants (52.32% male, mean age = 28.86). Measurements Compliance data, the proportion of responses to the study protocol, were extracted from each study alongside prompt frequency, total length of assessment period, substance use population and device used to administer EMA prompts. Findings The pooled compliance rate across all studies was 75.06% [95% confidence interval (CI) = 72.37%, 77.65%]. There was no evidence that compliance rates were significantly associated with prompt frequency [Q(3) = 7.35, P = 0.061], length of assessment period [Q(2) = 2.40, P = 0.301], substance type [Q(3) = 6.30, P = 0.098] or device administration [Q(4) = 4.28, P = 0.369]. However, dependent samples (69.80%) had lower compliance rates than non‐dependent samples [76.02%; Q(1) = 4.13, P = 0.042]. Conclusions The pooled compliance rate for Ecological Momentary Assessment studies in substance‐using populations from 1998 to 2017 was lower than the recommended rate of 80%, and was not associated with frequency or duration of assessments.

The effects of sampling frequency and questionnaire length on perceived burden, compliance, and careless responding in experience sampling data in a student population

“…The current BURDEN, COMPLIANCE & CARELESS RESPONDING IN ESM DATA results suggest that these variations in questionnaire lengths can have a measurable impact on burden, data quality, and quantity. Most previous analyses have failed to find an association between questionnaire length and compliance (Jones et al, 2018;Ono et al, 2019;Soyster et al, 2019;Vachon et al, 2019), however some of these findings were limited by the availability of information on full questionnaire lengths. Moreover, while these analyses did control for the effects of some other design characteristics of the included studies, more subtle aspects (such as the detail of instructions, which might be adapted to the protocol intensity in individual studies) were not included.…”

“…Moreover, meta-analyses on predictors of compliance have come to varying conclusions. While Morren and colleagues (2009) found that shorter questionnaires were associated with better compliance, two pooled data analyses (Rintala, Wampers, Myin-Germeys, & Viechtbauer, 2018;Soyster et al, 2019) and other metaanalyses have not detected an association between questionnaire length and compliance (Jones et al, 2018;Ono et al, 2019;Vachon et al, 2019). However, findings from these analyses have to be interpreted cautiously, since they faced limitations such as low variability in included questionnaire lengths (Rintala et al, 2018) and lack of sufficient information on questionnaire length or compliance in reviewed articles (Jones et al, 2018;Ono et al, 2019;Vachon et al, 2019).…”

Impaired subjective self-control in alcohol use: An ecological momentary assessment study

“…Another limitation that should be mentioned is the relatively low compliance rate of 47.4% on the random EMA assessments. A recent meta-analysis showed that the average compliance rate for EMA studies in the substance use field is 75.1% (Jones et al, 2019). In the current study, we did not reimburse participants which may have led to lower compliance rate.…”

“…In the current study, we did not reimburse participants which may have led to lower compliance rate. The findings from the meta-analysis, however, suggest that possible moderators such as reimbursement and assessment burden, do not influence compliance rate (Jones et al, 2019). To reduce the risk of noncompliance due to competing priorities, participants were allowed up to 90 min to complete the random assessments.…”

A Roadmap for Integrating Neuroscience Into Addiction Treatment: A Consensus of the Neuroscience Interest Group of the International Society of Addiction Medicine

“…Test compliance, or lack thereof, seems to be problematic. A recent meta-analysis suggested that the compliance rate for EMA (the standard paradigm to administer mobile cognitive testing) with SUD samples was below the recommended rate of 80% (63). Designs including participant-initiated event-based assessments were associated with test compliance issues, whereas duration and frequency of assessment were not.…”