A Cross-Study Analysis of Mobile EMA in Monitoring Behavior and Well-Being: Insights to Refine EMA Methods (Preprint)

Abstract

BACKGROUND Ecological momentary assessment (EMA) offers an effective method to collect frequent, real-time data on an individual’s well-being. However, challenges exist in response consistency, completeness, and accuracy. OBJECTIVE This goal of this study is to analyze EMA responses across various settings, using data from multiple diverse studies to enhance the generalizability of conclusions. We analyze the influence of contextual factors on participant engagement with EMA prompts and inform improvements in EMA methodology. METHODS Data from 521 participants in nine clinical studies were analyzed using statistical and machine learning techniques. Data were collected with an in-house app on smartwatches or tablets for daily EMA responses. The analysis focused on response rate, completeness, quality, and alignment with sensor-observed behavior. RESULTS The average response rate (RR) was 79.04%. Participants were most responsive in the evening (75.41%) and on weekdays (70.22%), though results varied based on study demographics. RR correlation with the number of EMA questions at each session was r=-.335 (P<.001). RR correlations were also observed with sensor-detected activity level (r=.045, P<.001), being at home (r=.174, P<.001), and nearness to change points (r=.124, P<.001). In terms of response quality, the percentage of careless responses increased by .022 throughout the study (P<.001) and variance decreased by .363 (P<.001). For all analyses, differences in results were observed between the individual studies. CONCLUSIONS EMA response patterns are significantly influenced by participant demographics and study parameters. Tailoring EMA prompt strategies to specific participant characteristics can improve response rates and quality. Additionally, incorporating sensor-based behavior context enhances the effectiveness of EMA studies.