Small Effects: The Indispensable Foundation for a Cumulative Psychological Science

Abstract

We draw on genetics research to argue that complex psychological phenomena are most likely determined by a multitude of causes and that any individual cause is likely to have only a small effect. Building on this, we highlight the dangers of a publication culture that continues to demand large effects. First, it rewards inflated effects that are unlikely to be real and encourages practices likely to yield such effects. Second, it overlooks the small effects that are most likely to be real, hindering attempts to identify and understand the actual determinants of complex psychological phenomena. We then explain the theoretical and practical relevance of small effects, which can have substantial consequences, especially when considered at scale and over time. Finally, we suggest ways in which scholars can harness these insights to advance research and practices in psychology (i.e., leveraging the power of big data, machine learning, and crowdsourcing science; promoting rigorous preregistration, including prespecifying the smallest effect size of interest; contextualizing effects; changing cultural norms to reward accurate and meaningful effects rather than exaggerated and unreliable effects). Only once small effects are accepted as the norm, rather than the exception, can a reliable and reproducible cumulative psychological science be built.

Going beyond simplicity: Using machine learning to predict belief in conspiracy theories

“…Most importantly, the present results support the existence of more complex interactions and non-linear relationships between predictors of CT belief than previous research suggested, indicated by the RF prediction performance and ALE plots, which draws on similar findings in this research area (Imhoff et al, 2021). Further, the PFI revealed multiple psychological factors to be relevant for CT belief prediction in an integrated fashion, supporting the existence of small, interplaying effects as noted in Götz et al (2021). Resultingly, belief in CTs is best being predicted through multiple, complexly interwoven psychological factors, which underlines the need for extensive and novel analysis strategies in CT research.…”

“…Finally, most studies only test single candidate factors for CT belief at a time, ignoring interactions and covariates. This can be problematic, since complex psychological phenomena likely have many interplaying small-effect causes rather than few big ones (Götz et al, 2021). Arguably, belief in CTs falls into this area, since individual beliefs can be influenced by a variety of proximal and distal factors (e.g., Wyer Jr & Albarracín, 2005).…”

Preparing for Future Adversities: Lessons From the COVID-19 Pandemic in Australia for Promoting Relational Resilience in Families

“…Effect sizes were strongest for exposures in the more proximal postpartum period, as might be expected for factors closer in time ( 60 ). The smaller effects we observed are of public health interest, given that young adult assessment occurred 14 years prior to assessment during the pandemic and that relationship quality and social support are multi-determined ( 60 , 61 ).…”

The Misinformation Susceptibility Test (MIST): A psychometrically validated measure of news veracity discernment

“…In line with the best practices for scale development to recruit at least 300 participants per sample (Boateng et al, 2018;Clark & Watson, 1995Comrey & Lee, 1992;Guadagnoli & Velicer, 1988), as well as being highly powered (power = .90, α = .05) to detect the smallest effect size of interest (r = .10, needed N = 1,046; Anvari & Lakens, 2021;Funder & Ozer, 2019;Götz, Gosling, et al, 2021), Sample 2A, 2C, and 2D exceed the size requirements. Sample 2B was highly powered (power = .90, α = .05) to detect effect sizes r of .15 (needed N = 463).…”

“…Since the MIST-20 only takes about 2 minutes to complete, we recommend researchers to use the MIST-20 whenever possible. Third, while we were sufficiently powered to detect effect sizes similar to the original evaluation of the intervention (Roozenbeek & van der Linden, 2019), we did not have sufficient power to detect smaller nuances (Anvari & Lakens, 2021;Funder & Ozer, 2019;Götz, Gosling, et al, 2021).…”