A qualitative approach to guide choices for designing a diary study

Abstract

BackgroundElectronic diaries are increasingly used in diverse disciplines to collect momentary data on experienced feelings, cognitions, behavior and social context in real life situations. Choices to be made for an effective and feasible design are however a challenge. Careful and detailed documentation of argumentation of choosing a particular design, as well as general guidelines on how to design such studies are largely lacking in scientific papers. This qualitative study provides a systematic overview of arguments for choosing a specific diary study design (e.g. time frame) in order to optimize future design decisions.MethodsDuring the first data assessment round, 47 researchers experienced in diary research from twelve different countries participated. They gave a description of and arguments for choosing their diary design (i.e., study duration, measurement frequency, random or fixed assessment, momentary or retrospective assessment, allowed delay to respond to the beep). During the second round, 38 participants (81%) rated the importance of the different themes identified during the first assessment round for the different diary design topics.ResultsThe rationales for diary design choices reported during the first round were mostly strongly related to the research question. The rationales were categorized into four overarching themes: nature of the variables, reliability, feasibility, and statistics. During the second round, all overarching themes were considered important for all diary design topics.ConclusionsWe conclude that no golden standard for the optimal design of a diary study exists since the design depends heavily upon the research question of the study. The findings of the current study are helpful to explicate and guide the specific choices that have to be made when designing a diary study.Electronic supplementary materialThe online version of this article (10.1186/s12874-018-0579-6) contains supplementary material, which is available to authorized users.

The effects of sampling frequency and questionnaire length on perceived burden, compliance, and careless responding in experience sampling data in a student population

“…Interestingly, momentary burden was the highest in the three-beep conditions. This could indicate that frequent beeps might even be perceived as less burdensome in some cases, possibly because the higher predictability allows participants to better integrate the measures into their daily routine (as has previously been suggested by Janssens et al, 2018). This strongly contrasts with previous studies, where participants expressed a preference for lower sampling frequencies (Rosenkranz et al, 2019;Spook et al, 2013).…”

“…In an ESM study, participants are typically asked to fill in several short questionnaires at random times during the day over several days. Setting up an ESM study requires making numerous design choices (Janssens, Bos, Rosmalen, Wichers, & Riese, 2018). Many of these choices are thought to be associated with a trade-off between the wish to collect as much information as possible on the one hand, and the risk of compromising data quantity and quality as a result of the intensive assessment on the other (Arslan, Reitz, Driebe, Gerlach, & Penke 2019;May, Junghaenel, Ono, & Stone, 2018).…”

Making the black box transparent: A template and tutorial for (pre-)registration of studies using Experience Sampling Methods (ESM)

“…Wide variation exists in the number of days over which ESM data are collected, as a function of expected variability in target behaviors and feasibility (Janssens, Bos, Rosmalen, Wichers, & Riese, 2018). Occasionally, researchers wish to extend the ESM period for (a subsample of) participants.…”

“…Many reports of ESM studies only include a description of variables that were analyzed for that specific study. While the number of items per ESM assessment varies greatly (Janssens et al, 2018), the total number and type of items included in the ESM questionnaire are only infrequently reported (Morren, van Dulmen, Ouwerkerk, & Bensing, 2009;Vachon, et al, 2019;van Roekel, Keijsers, & Chung, 2019). Here, researchers are asked to provide a general description of the total questionnaire length.…”

The pedagogical research project - an essential tool for the development of research competencies in the field of education

“…Depending on the type of research, of intentionality, both quantitative and qualitative methods can be used (Astalin, 2013;White & Sabarwal, 2014;McKim, 2017;Mohajan, 2018;Janssens, Bos, Rosmalen, Wichers & Riese, 2018). The two paradigms, that of research based on quantitative methods and the other on qualitative methods, must not necessarily be seen in opposition but in complementarity.…”