Empirically derived guidelines for effect size interpretation in social psychology

Abstract

This study estimates empirically derived guidelines for effect size interpretation for research in social psychology overall and subdisciplines within social psychology, based on analysis of the true distributions of the two types of effect size measures widely used in social psychology (correlation coefficient and standardized mean differences). Analysis of empirically derived distributions of 12,170 correlation coefficients and 6,447 Cohen's d statistics extracted from studies included in 134 published meta-analyses revealed that the 25th, 50th, and 75th percentiles corresponded to correlation coefficient values of 0.12, 0.24, and 0.41 and to Cohen's d values of 0.15, 0.36, and 0.65 respectively. The analysis suggests that the widely used Cohen's guidelines tend to overestimate medium and large effect sizes. Empirically derived effect size distributions in social psychology overall and subdisciplines can be used both for effect size interpretation and for sample size planning when other information about effect size is not available.

Research on climate change in social psychology publications: A systematic review

“…That is, when a climate-changerelated study was published in a social psychological journal, we can safely assume that it has social psychological substance, both because the authors chose to submit it to a social psychological journal in the first place and because the reviewers and editors chose to accept it to be published in the journal. This method has been used in other reviews or bibliometric analyses of social psychological publications (Chester & Lasko, 2019;Haslam & Kashima, 2010;Lovakov & Agadullina, 2021).…”

“…2 We defined social psychological journals as the 63 journals that were indexed under "Psychology, social" category in Journal Citation Reports 2019, Web of Science. This definition has been used by other researchers (Haslam & Kashima, 2010;Lovakov & Agadullina, 2021). We performed a search within each journal on the PsycInfo database (on 3 June 2020) using the search term "climate change OR global warming" in the TX field.…”

Denouncing the use of field-specific effect size distributions to inform magnitude

“…In addition to the Hemphill (2003) study discussed above, we identified four other studies from the field of psychology that proposed ES benchmark value recommendations based on the observed ES values distribution approach, namely Gignac & Szodorai (2016), Rubio-Aparicio et al (2018), Lovakov & Agadullina (2017), and Brydges (2019) (Table 1). These are particularly influential because they have been published in prospering subfields of psychology, including personality and individual differences, clinical psychology, social psychology, and gerontology.…”

“…Lovakov & Agadullina (2017) gathered 161 meta-analyses among 29 social psychology journals and reported the lower, middle, and upper thirds of Pearson's r. They recommended ES values between r = 0 and r = 0.1 to be considered a small effect, between r = 0.1 and r = 0.25 to be considered a medium effect, and between r = 0.25 and r = 0.4 to be considered a large effect (Lovakov & Agadullina, 2017), highlighting that Cohen (1988) guidelines were too conservative.…”

What can the implicit social cognition literature teach us about implicit social cognition?

“…The finding that the relationship between explicit attitudes and intergroup behavior is almost exactly the same size (Kurdi et al, 2019b) receives no mention. What's more, the mean implicit-behavior correlation sits right around the 25 th percentile of all effect sizes in social psychology, with Running Head: THE STATE OF THE IMPLICIT SOCIAL COGNITION LITERATURE 5 the largest implicit-behavior correlations at the individual level approaching the 70 th percentile of that distribution (Lovakov & Agadullina, 2021).…”