Simpson's paradox in psychological science: a practical guide

Abstract

The direction of an association at the population-level may be reversed within the subgroups comprising that population—a striking observation called Simpson's paradox. When facing this pattern, psychologists often view it as anomalous. Here, we argue that Simpson's paradox is more common than conventionally thought, and typically results in incorrect interpretations—potentially with harmful consequences. We support this claim by reviewing results from cognitive neuroscience, behavior genetics, clinical psychology, personality psychology, educational psychology, intelligence research, and simulation studies. We show that Simpson's paradox is most likely to occur when inferences are drawn across different levels of explanation (e.g., from populations to subgroups, or subgroups to individuals). We propose a set of statistical markers indicative of the paradox, and offer psychometric solutions for dealing with the paradox when encountered—including a toolbox in R for detecting Simpson's paradox. We show that explicit modeling of situations in which the paradox might occur not only prevents incorrect interpretations of data, but also results in a deeper understanding of what data tell us about the world.

Social Media Use, Social Media Stress, and Sleep: Examining Cross-Sectional and Longitudinal Relationships in Adolescents

“…However, at the between-person level we found a positive relationship between SM use and daytime sleepiness. Together, these findings may reflect a Simpson's Paradox (i.e., findings at the withinperson level are the opposite of findings at the between-person level; Kievit, Frankenhuis, Waldorp, & Borsboom, 2013). While boys who more frequently use SM reported a longer sleep latency and daytime sleepiness than their peers (between-person level), boys reported less daytime sleepiness in periods when they more frequently use SM compared to periods when they less frequently use SM (within-person level).…”

Disgust sensitivity relates to moral foundations independent of political ideology.

“…Consistent with Tybur et al (2010), conservatism showed a small positive correlation with sexual Disgust sensitivity and moral judgments 9 disgust sensitivity, but showed almost no correlation with moral and pathogen disgust sensitivity. Participants were nested within samples, and failing to account for this nesting could yield spurious correlations (Kievit, Frankenhuis, Waldorp, & Borsboom, 2013). For example, four samples may on average have low disgust sensitivity and low moral judgment, and four samples may on average have high disgust sensitivity and high moral judgment.…”

Adolescents’ autonomous functioning and implicit theories of ability as predictors of their school achievement and week-to-week study regulation and well-being

“…Our proposition aims to draw researchers' attention to a likely ''Simpson's paradox". According to Simpson paradox, an association that is observed in different sub-groups (say, a negative week-toweek relation between variables A and B at the intrapersonal level) may be reversed when the data are pooled (i.e., the negative intrapersonal relation between variables A and B may become positive when the data are aggregated at the student level) (Kievit, Frankenhuis, Waldorp, & Borsboom, 2013). Using ''Simpson's paradox" as a springboard, we are wondering whether the direction of an association at the interpersonal-level (when people are asked to provide summary accounts of their psychological states and experiences -see Bolger et al, 2003) may sometimes be reversed (or be absent) at the intrapersonal level (when we assess the same constructs in a more dynamic, day-to-day way).…”