Addressing two blind spots of commonly used experimental designs: The highly-repeated within-person approach

Abstract

Two well documented but still neglected blind spots of often-used study designs limit a researcher’s ability to make inferences about psychological phenomenon. First, typical designs focus on effects of conditions at the group level and are not able to assess the extent to which effects characterize each participant in the study. This blind spot can lead to erroneous (or incomplete) conclusions about the effects of manipulations both for a given participant and at the group level. Second, commonly used research designs often use only a limited sample of stimuli, constraining conclusions to the particular stimuli. This blind spot can lead to non-replication when different stimuli are used. We propose that the Highly-Repeated Within-Person (HRWP) approach helps mitigate these limitations. Using a study on the effects of anti-smoking messages, we illustrate how the HRWP approach helps alert researchers when the conclusions at the group-level may not apply to all (or any) participant, quantifies the heterogeneity of effects of manipulations across people, and increases confidence regarding the generalizability of the effects. We discuss how the HRWP approach may help conceptualize issues of replicability in a new light.