Ecological Momentary Assessment in Behavioral Research: Addressing Technological and Human Participant Challenges

Abstract

BackgroundEcological momentary assessment (EMA) assesses individuals’ current experiences, behaviors, and moods as they occur in real time and in their natural environment. EMA studies, particularly those of longer duration, are complex and require an infrastructure to support the data flow and monitoring of EMA completion.ObjectiveOur objective is to provide a practical guide to developing and implementing an EMA study, with a focus on the methods and logistics of conducting such a study.MethodsThe EMPOWER study was a 12-month study that used EMA to examine the triggers of lapses and relapse following intentional weight loss. We report on several studies that informed the implementation of the EMPOWER study: (1) a series of pilot studies, (2) the EMPOWER study’s infrastructure, (3) training of study participants in use of smartphones and the EMA protocol and, (4) strategies used to enhance adherence to completing EMA surveys.ResultsThe study enrolled 151 adults and had 87.4% (132/151) retention rate at 12 months. Our learning experiences in the development of the infrastructure to support EMA assessments for the 12-month study spanned several topic areas. Included were the optimal frequency of EMA prompts to maximize data collection without overburdening participants; the timing and scheduling of EMA prompts; technological lessons to support a longitudinal study, such as proper communication between the Android smartphone, the Web server, and the database server; and use of a phone that provided access to the system’s functionality for EMA data collection to avoid loss of data and minimize the impact of loss of network connectivity. These were especially important in a 1-year study with participants who might travel. It also protected the data collection from any server-side failure. Regular monitoring of participants’ response to EMA prompts was critical, so we built in incentives to enhance completion of EMA surveys. During the first 6 months of the 12-month study interval, adherence to completing EMA surveys was high, with 88.3% (66,978/75,888) completion of random assessments and around 90% (23,411/25,929 and 23,343/26,010) completion of time-contingent assessments, despite the duration of EMA data collection and challenges with implementation.ConclusionsThis work informed us of the necessary preliminary steps to plan and prepare a longitudinal study using smartphone technology and the critical elements to ensure participant engagement in the potentially burdensome protocol, which spanned 12 months. While this was a technology-supported and -programmed study, it required close oversight to ensure all elements were functioning correctly, particularly once human participants became involved.

Compliance with ecological momentary assessment protocols in substance users: a meta‐analysis

“…We demonstrated no evidence that the number of prompt frequency influenced compliance rates, which contradicts recommendations by Burke et al [3], who suggest that random prompts should be limited to a maximum of five per day in order to reduce participant burden. We demonstrated no evidence that the number of prompt frequency influenced compliance rates, which contradicts recommendations by Burke et al [3], who suggest that random prompts should be limited to a maximum of five per day in order to reduce participant burden.…”

“…2 demonstrates that the greatest frequency of compliance rates is between 80 and 85%. 3 Finally, it is possible that our coding for some of our moderators such as training and reimbursement was not sensitive, i.e. Secondly, we identified numerous studies that we could not include in our analyses due to an absence of compliance data reported.…”

“…We excluded EMA studies which only examined event-contingent assessments, as it is not possible to report compliance to self-initiated reports [3]. We excluded EMA studies which only examined event-contingent assessments, as it is not possible to report compliance to self-initiated reports [3].…”

“…EMA methods provide patterns of rich data which can model relationships between variables over time in a way that cross-sectional assessment cannot. The popularity of EMA methods has increased rapidly in recent years [2], owing in part to the development of mobile technology and software packages that enable sophisticated data analyses [3,4]. A typical EMA study may send participants a number of signalled prompts on a portable device.…”

A Novel Approach to Evaluating Mobile Smartphone Screen Time for iPhones: Feasibility and Preliminary Findings (Preprint)

“…In these approaches, participants are typically contacted multiple times per day to report their real-time screen use [15,16]. While this approach improves upon self-report biases, such as recall bias, this approach can be highly burdensome to participants [17]. In addition, these methodologies often fail to obtain large-scale and representative samples due to costs to researchers to provide the compensation necessary to attract participants [18].…”