The effects of sampling frequency and questionnaire length on perceived burden, compliance, and careless responding in experience sampling data in a student population

Eisele, Gudrun; Vachon, Hugo; Lafit, Ginette; Kuppens, Peter; Houben, Marlies; Myin-Germeys, Inez; Viechtbauer, Wolfgang

doi:10.31234/osf.io/zf4nm

Cited by 53 publications

(48 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, the reported 78.7% of completed assessments in this study is a strong indication for the feasibility of ESM in a majority (18/21, 85%) of our MCI sample. A high sampling frequency, which was applied in this study with 8 beeps per day, is not thought to hinder ESM use, while the length of the questionnaires can increase burden [ 39 ]. The overall positive participants’ feedback on the procedure, including the length and frequency, supports the chosen ESM set-up.…”

Section: Discussionmentioning

confidence: 99%

Smartphone-Based Experience Sampling in People With Mild Cognitive Impairment: Feasibility and Usability Study

Bartels¹,

Knippenberg²,

Malinowsky³

et al. 2020

JMIR Aging

View full text Add to dashboard Cite

Background Daily functioning of people with cognitive disorders such as mild cognitive impairment (MCI) is usually depicted by retrospective questionnaires, which can be memory-biased and neglect fluctuations over time or contexts. Objective This study examines the feasibility and usability of applying the experience sampling method (ESM) in people with MCI to provide a detailed and dynamic picture of behavioral, emotional, and cognitive patterns in everyday life. Methods For 6 consecutive days, 21 people with MCI used an ESM app on their smartphones. At 8 semi-random timepoints per day, participants filled in momentary questionnaires on mood, activities, social context, and subjective cognitive complaints. Feasibility was determined through self-reports and observable human-technology interactions. Usability was demonstrated on an individual and group level. Results Of the 21 participants, 3 dropped out due to forgetting to carry their smartphones or forgetting the study instructions. In the remaining 18 individuals, the compliance rate was high, at 78.7%. Participants reported that momentary questions reflected their daily experiences well. Of the 18 participants, 13 (72%) experienced the increase in awareness of their own memory functions as pleasant or neutral. Conclusions Support was found for the general feasibility of smartphone-based experience sampling in people with MCI. However, many older adults with MCI are currently not in possession of smartphones, and study adherence seems challenging for a minority of individuals. Momentary data can increase the insights into daily patterns and may guide the person-tailored development of self-management strategies in clinical settings.

show abstract

Section: Discussionmentioning

confidence: 99%

Smartphone-Based Experience Sampling in People With Mild Cognitive Impairment: Feasibility and Usability Study

Bartels¹,

Knippenberg²,

Malinowsky³

et al. 2020

JMIR Aging

View full text Add to dashboard Cite

show abstract

“…Hofmans and colleagues (2019) outline the whole process of conducting an ESM study, including set-up and data analysis. Finally, some publications focus on narrower issues, such as the optimal ESM schedule (that is, when participants receive their daily invitations; van Berkel et al, 2019), sampling frequency and length of the survey (Eisele et al, 2020), the instructions to be used (Stone, Wen, Schneider, & Junghaenel, 2020), the estimation of reliability (Nezlek, 2017;Schönbrodt, Zygar, Nestler, Pusch, & Hagemeyer, submitted), validity (Vogelsmeier, Vermunt, van Roekel, & De Roover, 2019), or the construction of state measures (Horstmann & Ziegler, in revision). However, there are not many that focus on the very practical issues of setting up an ESM study.…”

Section: Alternative Resourcesmentioning

confidence: 99%

Experience Sampling and Daily Diary Studies: Basic Concepts, Designs, and Challenges.pdf

Horstmann¹

2020

Preprint

View full text Add to dashboard Cite

Experience sampling and daily diary methods have become increasingly popular among psychologists. The repeated assessment of persons in their daily lives allows capturing how a person feels, thinks, or behaves or what he or she desires in the very moment. The current chapter describes basic concepts of experience sampling studies, gives an overview of possible designs and challenges that may be encountered when setting up the first experience sampling study. To overcome these challenges, three basic questions can be answered: (A) What is the construct being measured? (B) What is the purpose of the measure? (C) What is the targeted population of persons and situations? Finally, practical advice is given on how to think through and pilot test an experience sampling study before data collection begins.

show abstract

“…Here, researchers are asked to provide a general description of the total questionnaire length. A longer ESM questionnaire, especially one with more open-ended items, signifies a greater participant burden, which can reduce the compliance rate as well as the data quality (Eisele et al, 2020). Without knowledge of the total number of items, the potential effect of the total questionnaire length on the compliance rate is unclear.…”

Section: Total Number and Type Of Items (Open-ended Or Closed)mentioning

confidence: 99%

“…Low compliance and thus factors influencing compliance can reduce the quality of the data (Delespaul, 1995;Eisele et al, 2020;Palmier-Claus, et al, 2011;Rintala, et al, 2019). For pre-data collection pre-registration of ESM studies, we recommend researchers state decision rules related to participant dropouts (e.g., whether observations prior to the dropout will be included in the analysis).…”

Section: Compliancementioning

confidence: 99%

“…Encouragingly, recent years have seen a significant elevation in interest and research energy directed towards addressing methodological and statistical issues in ESM (e.g. Eisele et al, 2020;Himmelstein, Woods & Wright, 2019;Houben, Van Den Noortgate, & Kuppens, 2015;Rintala et al, 2018;Schuurman & Hamaker, 2019;Vachon, Viechtbauer, Rintala, & Myin-Germeys, 2019;Wright & Zimmermann, 2019). Some of these advances present ESM researchers with new choices, which all represent key decisions for a study and as such, potential points of variation in methodological and statistical approaches within as well as between studies.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Making the black box transparent: A template and tutorial for (pre-)registration of studies using Experience Sampling Methods (ESM)

Lafit

Achterhof

Hiekkaranta

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

A growing interest in understanding complex and dynamic psychological processes as they occur in everyday life has led to an increase in studies using Ambulatory Assessment techniques, including the Experience Sampling Method (ESM) and Ecological Momentary Assessment (EMA). There are, however, numerous “forking paths” and researcher degrees of freedom, even beyond those typically encountered with other research methodologies. Whilst a number of researchers working with ESM techniques are actively engaged in efforts to increase the methodological rigor and transparency of such research, currently, there is little routine implementation of open science practices in ESM research. In the current paper, we discuss the ways in which ESM research is especially vulnerable to threats to transparency, reproducibility and replicability. We propose that greater use of (pre-)registration, a cornerstone of open science, may address some of these threats to the transparency of ESM research. (Pre-)registration of ESM research is not without challenges, including model selection, accounting for potential model convergence issues and the use of pre-existing datasets. As these may prove to be significant barriers to (pre-)registration for ESM researchers, we also discuss ways of overcoming these challenges and of documenting them in a (pre-)registration. A further challenge is that current general templates do not adequately capture the unique features of ESM. Here we present a (pre-)registration template for ESM research, adapted from the original Pre-Registration Challenge (Mellor et al., 2019) and pre-registration of pre-existing data (van den Akker et al., 2020) templates, and provide examples of how to complete this.

show abstract

The effects of sampling frequency and questionnaire length on perceived burden, compliance, and careless responding in experience sampling data in a student population

Cited by 53 publications

References 10 publications

Smartphone-Based Experience Sampling in People With Mild Cognitive Impairment: Feasibility and Usability Study

Smartphone-Based Experience Sampling in People With Mild Cognitive Impairment: Feasibility and Usability Study

Experience Sampling and Daily Diary Studies: Basic Concepts, Designs, and Challenges.pdf

Making the black box transparent: A template and tutorial for (pre-)registration of studies using Experience Sampling Methods (ESM)

Contact Info

Product

Resources

About