Content validity and methodological considerations in ecological momentary assessment studies on physical activity and sedentary behaviour: a systematic review

Abstract

Background: Ecological momentary assessment (EMA) is a method of collecting real-time data based on repeated measures and observations that take place in participant's daily environment. EMA has many advantages over more traditional, retrospective questionnaires. However, EMA faces some challenges to reach its full potential. The aims of this systematic review are to (1) investigate whether and how content validity of the items (i.e. the specific questions that are part of a larger EMA questionnaire) used in EMA studies on physical activity and sedentary behaviour was assessed, and (2) provide an overview of important methodological considerations of EMA in measuring physical activity and sedentary behaviour. Methods: Thirty papers (twenty unique studies) were systematically reviewed and variables were coded and analysed within the following 4 domains: (1) Content validity, (2) Sampling approach, (3) Data input modalities and (4) Degree of EMA completion. Results: Only about half of the studies reported the specific items (n = 12) and the source of the items (n = 11). None of the studies specifically assessed the content validity of the items used. Only a minority (n = 5) of the studies reported any training, and one tested the comprehensibility of the EMA items. A wide variability was found in the design and methodology of the EMA. A minority of the studies (n = 7) reported a rationale for the used prompt frequency, time selection, and monitoring period. Retrospective assessment periods varied from 'now' to 'in the last 3.5 hours'. In some studies there was a possibility to delay (n = 6) or deactivate (n = 10) the prompt, and some provided reminders after the first prompt (n = 9). Conclusions: Almost no EMA studies reported the content validation of the items used. We recommend using the COSMIN checklist (COnsensus-based Standards for the selection of health Measurement INstruments) to report on the content validity of EMA items. Furthermore, as often no rationale was provided for several methodological decisions, the following three recommendations are made. First, provide a rationale for choosing the sampling modalities. Second, to ensure assessment 'in the moment', think carefully about the retrospective assessment period, reminders, and deactivation of the prompt. Third, as high completion rates are important for representativeness of the data and generalizability of the findings, report completion rates.

Physical activity in an air-polluted environment: behavioral, psychological and neuroimaging protocol for a prospective cohort study (Healthy Aging in Industrial Environment study – Program 4)

“…In the 4HAIE study, we employ ongoing surveillance of study adherence with a system of both automatic notifications and reminders (app) as well as telephone, SMS, and email check-ins and follow-up contacts when necessary. Our approach was designed to align with recommended strategies for optimizing response rates in EMA studies such as incorporating ongoing compliance monitoring, check-in strategies and providing compliance-based incentives to support participant engagement [ 99 , 100 ].…”

The impact of mental and somatic stressors on physical activity and sedentary behaviour in adults with type 2 diabetes mellitus: a diary study

“…This is in line with numerous diary studies (e.g., Bouwmans et al (2017)) and was done to limit the time needed to complete the diary and hence minimise the chances of drop-out or random responses. Additionally, considering the repeated assessment in daily life, traditional questionnaires assessing mental and somatic stressors are not by default valid for this type of research (Degroote et al, 2020). Participants were instructed to indicate how strongly they experienced the stressor "right now" using a 10-point scale, ranging from 1 "(absolutely not)" to 10 "(very much)".…”

Systematic review of ecological momentary assessment (EMA) studies of five public health-related behaviours: review protocol

“…Previous reviews of EMA studies have focused on clinical conditions such as borderline personality disorder, 3 psychotic disorder, 4 mood disorders, 5 binge eating, 6 bulimia nervosa, 7 anxiety disorder, 8 schizophrenia, 9 alcohol use disorder, 10 chronic pain 11 and specific populations such as children and adolescents, 12 youth 13 and older adults. 14 Health behaviour-specific reviews of EMA studies have focused on physical activity, 15 16 sedentary behaviour, 16 alcohol use, 17 craving and substance use, 18 dietary behaviours 19 and the relationship between alcohol use and sexual decision-making. 20 Previous EMA reviews have also focused on inter-relations between specific psychological variables, such as the association of everyday social interactions with intra-individual variability in affect.…”