Are Retrospective Assessments Means of People’s Experiences? Accounting for Interpersonal and Intrapersonal Variability when Comparing Retrospective Assessment Data to Ecological Momentary Assessment Data

Abstract

Retrospective Assessment (RA) scores are often found to be higher than the mean of Ecological Momentary Assessment (EMA) scores about a concurrent period. This difference is generally interpreted as bias towards salient experiences in RA. During RA, participants are often asked to summarize their experiences in unspecific terms, which may indeed facilitate bias. At least in this unspecific form, the summary that participants apply to their remembered experiences can take many different forms. In this study, we reanalyzed an existing dataset (N = 92) using a repeated N = 1 approach. We reported on interindividual differences between EMA data and RA score, and assessed for each participant whether it was likely that their RA score was an approximation of the mean of their experiences as captured by their EMA data. We found considerable interpersonal differences in the difference between EMA scores and RA scores, as well as some extreme cases. Furthermore, for a considerable part of the sample (n = 46 for positive affect, n = 60 for negative affect), we did not reject the null hypothesis that their RA score represented the mean of their experiences as captured by their EMA data. We conclude that in its current unspecific form, RA may facilitate bias, although not for everyone. Future studies may determine whether more specific forms of RA reduce bias, while acknowledging interindividual differences.

Measuring the Variability of Personality Traits With Interval Responses: Psychometric Properties of the Dual-Range Slider Response Format

“…Variability can either be assessed directly via an interval response in a retrospective rating of the respective quantity or frequency, or it can be estimated via the aggregation of repeated responses over a given time period (e.g., by assessing the variance of within-person observations in an EMA design). Leertouwer et al (2021) found that for a substantial proportion of respondents (50.0% for a positive-affect scale and 60.9% for a negative-affect scale), the retrospective assessment with a single-response format approximated the mean of longitudinal assessments. Fleeson and Gallagher (2009) reported moderate to high correlations (r = .42 to r = .56) between self-report questionnaires and means of EMA measures.…”

“…An ideal approach to investigating the convergent validity of longitudinal single-response measures and retrospective interval-response measures would be to conduct a longitudinal study using the EMA method as a reference for validation (e.g., Leertouwer et al, 2021). However, such an approach comes with substantial costs and challenges, especially when considering the lack of preliminary evidence regarding the validity of the interval-response format for measuring personality.…”

Understanding EMA Data: A Tutorial on Exploring Item Performance in Ecological Momentary Assessment Data

“…Recent work has compared momentary versus recalled affect (Greene et al, 2022;Leertouwer et al, 2021) and daily versus biweekly depressive symptoms (Horwitz et al, 2023). Example ways to study the predictive and concurrent validity for single-item measures can be found in Song et al (2022).…”

Equilibrium Causal Models: Connecting Dynamical Systems Modeling and Cross-Sectional Data Analysis

“…Of course, prior research has also shown that people's responses to such measurements can be influenced by contextual effects, such as one's current state (Augustine & Larsen, 2012;Barrett, 1997;Bower, 1981;Brose et al, 2013;Leertouwer et al, 2021), which means that, even if we attempt to measure the equilibrium or trait, we are likely not able to do so perfectly. For example, measurements of general life satisfaction appear to be influenced by the mood of the participant, and potentially even the weather on the day of measurement (Schwarz & Clore, 1983).…”