Experience Sampling Methodology

Abstract

Experience sampling is a method aimed primarily at examining within-individual covariation of transient phenomena utilizing repeated measures. It can be applied to test nuanced predictions of extant theories and can provide insights that are otherwise difficult to obtain. It does so by examining the phenomena of interest close to where they occur and thus avoiding issues with recall and similar concerns. Data collected through the experience sampling method (ESM) can, alternatively, be utilized to collect highly reliable data to investigate between-individual phenomena. A number of decisions need to be made when designing an ESM study. Study duration and intensity (that is, total days of measurement and total assessments per day) represent a tradeoff between data richness and participant fatigue that needs to be carefully weighed. Other scheduling options need to be considered, such as triggered versus scheduled surveys. Researchers also need to be aware of the generally high potential cost of this approach, as well as the monetary and nonmonetary resources required. The intensity of this method also requires special consideration of the sample and the context. Proper screening is invaluable; ensuring that participants and their context is applicable and appropriate to the design is an important first step. The next step is ensuring that the surveys are planned in a compatible way to the sample, and that the surveys are designed to appropriately and rigorously collect data that can be used to accomplish the aims of the study at hand. Furthermore, ESM data typically requires proper consideration in regards to how the data will be analyzed and how results will be interpreted. Proper attention to analytic approaches (typically multilevel) is required. Finally, when interpreting results from ESM data, one must not forget that these effects typically represent processes that occur continuously across individuals’ working lives—effect sizes thus need to be considered with this in mind.

Walking on eggshells: A self-control perspective on workplace political correctness.

“…For Study 1, ESM effect sizes tend to be small (based on a combination of the analysis focusing on within-individual variance only, combined with the short time frames [e.g., several hours] over which relationships are investigated; Lin, Chang, et al, 2021; Lin, Poulton, et al, 2021; Puranik et al, 2021; Shockley et al, 2021; Tang, Ilies, et al, 2021). Thus, we agree with Gabriel et al (2019, p. 976) who warn “against automatic negative reactions to small within-person effect sizes” in ESM studies because while the effects may appear small, they also manifest within a short amount of time (see also Koopman & Dimotakis, in press). Plus, relationships in the experimental studies were based on a single episode such as a recalled event or a writing task, which should have smaller effects.…”

From free pastures to penned in: The within-person effects of psychological reactance on side-hustlers’ hostility and initiative in full-time work.

“…In line with Study 1, these 431 observations constitute complete employee data points, wherein participants provided a Thursday survey and Friday survey for the same week. As in Study 1, we utilized FIML to address missing data on our prior-day controls, which were modeled with fixed effects per prior research (Koopman & Dimotakis, 2022). Employees received $5 for joining the study, $4 for each completed survey, and a $5 bonus for completing 4 or more surveys.…”

“…1To promote the inclusion of coworkers with daily opportunities to observe focal employees, we required Study 1 focal participants to nominate a coworker with whom they interact daily. To ensure the effectiveness of this recommended inclusion criterion (Koopman & Dimotakis, 2022), we asked coworkers to rate daily opportunities to observe focal employees using the three-item measure from Judge and Ferris (1993). Over the 3-week study, coworkers’ daily mean rating of opportunity to observe the focal employee was 5.89 out of 7 ( SD = 1.07).…”

When, why, and for whom is receiving help actually helpful? Differential effects of receiving empowering and nonempowering help based on recipient gender.

“…However, small effect sizes are common in such studies in general (e.g., Koopman et al, 2022; Lin et al, 2021; Sabey et al, 2021), and for this study specifically it is important to keep in mind that the variance we explain reflects approximately 4 hr of a person’s day. As employees spend nearly 2,000 hr per year at work, even small changes over a brief window could be impactful over a longer period of time (Abelson, 1985; Funder & Ozer, 2019; Koopman & Dimotakis, in press). A related issue is that the effect size for competence in our ESM study is smaller than our other controlled mechanisms.…”