Effect size reporting and interpretation in social personality research

Farmus, Linda; Beribisky, Nataly; Gutierrez, Naomi Martinez; Alter, Udi; Panzarella, Emily; Cribbie, Robert A.

doi:10.1007/s12144-021-02621-7

Cited by 4 publications

(6 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This requirement manifests in the aforementioned "inefficiency," but also warrants us to have less uncertainty in our effect size estimates and conclusions. Importantly, focusing on the magnitude of effects, precision, and uncertainty beyond the decision to reject or not to reject the null hypothesis is in line with recommended practices (e. g., Amrhein et al, 2019;Cumming, 2012Cumming, , 2014Farmus et al, 2022;Fidler & Loftus, 2009). In fact, we strongly encourage readers to take stock of the magnitude of the relationship between a predictor and outcome, its precision, and the proportion and location of the confidence band in relation to the SESOI region, regardless of the NHST decision.…”

Section: Why Negligible Effect Tests Should Require Larger Sample Sizes?mentioning

confidence: 63%

“…The Quantitative Methods for Psychology are the most prevalent effect size reported in psychological research (Farmus et al, 2022) and are often the default output from statistical software packages. Similarly, the standard error provided by software output and reported in published articles is almost always tied to the unstandardized effect.…”

Section: Testing For Negligible Association Using Equivalence Testsmentioning

confidence: 99%

“…Using unstandardized regression coefficients as an effect size is both strongly recommended (e. g., Pek & Flora, 2018) and most commonly implemented in psychology (Farmus et al, 2022). Unstandardized regression coefficients and their associated standard errors are also usually the default output from most statistical software and therefore the easiest to retrieve.…”

Section: More About Using Standardized or Unstandardized Effectsmentioning

confidence: 99%

See 2 more Smart Citations

Determining Negligible Associations in Regression

Alter¹,

Counsell²

2023

TQMP

Self Cite

View full text Add to dashboard Cite

Psychological research is rife with inappropriately concluding "no effect" between predictors and outcome in regression models following statistically nonsignificant results. However, this approach is methodologically flawed because failing to reject the null hypothesis using traditional, difference-based tests does not mean the null is true. Using this approach leads to high rates of incorrect conclusions that flood psychological literature. This paper introduces a novel, methodologically sound alternative. In this paper, we demonstrate how an equivalence testing approach can be applied to multiple regression (which we refer to here as "negligible effect testing") to evaluate whether a predictor (measured in standardized or unstandardized units) has a negligible association with the outcome. In the first part of the paper, we evaluate the performance of two equivalence-based techniques and compare them to the traditional, difference-based test via a Monte Carlo simulation study. In the second part of the paper, we use examples from the literature to illustrate how researchers can implement the recommended negligible effect testing methods in their own work using open-access and user-friendly tools (negligible R package and Shiny app). Finally, we discuss how to report and interpret results from negligible effect testing and provide practical recommendations for best research practices based on the simulation results. All materials, including R code, results, and additional resources, are publicly available on the Open Science Framework (OSF): osf.io/w96xe/.

show abstract

Section: Why Negligible Effect Tests Should Require Larger Sample Sizes?mentioning

confidence: 63%

Section: Testing For Negligible Association Using Equivalence Testsmentioning

confidence: 99%

Section: More About Using Standardized or Unstandardized Effectsmentioning

confidence: 99%

See 1 more Smart Citation

Determining Negligible Associations in Regression

Alter¹,

Counsell²

2023

TQMP

Self Cite

View full text Add to dashboard Cite

show abstract

“…have requested authors to discuss sample size determination for some years now. With this emphasis has come an appreciation that further work is needed, because researchers sometimes show confusion or disagreement about the starting effect size needed to make decisions from a priori power analysis (see Blake & Gangestad, 2020;Farmus et al, 2023).…”

Section: Journal Of Experimental Social Psychologymentioning

confidence: 99%

Power to Detect What? Considerations for Planning and Evaluating Sample Size

Giner-Sorolla,

Montoya,

Reifman

et al. 2024

Pers Soc Psychol Rev

View full text Add to dashboard Cite

Academic Abstract In the wake of the replication crisis, social and personality psychologists have increased attention to power analysis and the adequacy of sample sizes. In this article, we analyze current controversies in this area, including choosing effect sizes, why and whether power analyses should be conducted on already-collected data, how to mitigate the negative effects of sample size criteria on specific kinds of research, and which power criterion to use. For novel research questions, we advocate that researchers base sample sizes on effects that are likely to be cost-effective for other people to implement (in applied settings) or to study (in basic research settings), given the limitations of interest-based minimums or field-wide effect sizes. We discuss two alternatives to power analysis, precision analysis and sequential analysis, and end with recommendations for improving the practices of researchers, reviewers, and journal editors in social-personality psychology. Public Abstract Recently, social-personality psychology has been criticized for basing some of its conclusions on studies with low numbers of participants. As a result, power analysis, a mathematical way to ensure that a study has enough participants to reliably “detect” a given size of psychological effect, has become popular. This article describes power analysis and discusses some controversies about it, including how researchers should derive assumptions about effect size, and how the requirements of power analysis can be applied without harming research on hard-to-reach and marginalized communities. For novel research questions, we advocate that researchers base sample sizes on effects that are likely to be cost-effective for other people to implement (in applied settings) or to study (in basic research settings). We discuss two alternatives to power analysis, precision analysis and sequential analysis, and end with recommendations for improving the practices of researchers, reviewers, and journal editors in social-personality psychology.

show abstract

“…For example, the Journal of Personality and Social Psychology now requires authors to address "justifiable power consideration" (American Psychological Association, 2022; see also Leach, 2021), while other journals such as Personality and Social Psychology Bulletin (Crandall et al, 2018), Social and Personality Psychological Science (Vazire, 2016), and Journal of Experimental Social Psychology (Giner-Sorolla, 2016), have requested authors to discuss sample size determination for some years now. With this emphasis has come an appreciation that further work is needed, because researchers sometimes show confusion or disagreement about the starting effect size needed to make decisions from a priori power analysis (see Blake & Gangestad, 2020;Farmus et al, 2023).…”

Section: Power To Detect What? Considerations For Planning and Evalua...mentioning

confidence: 99%

Power to Detect What? Considerations for Planning and Evaluating Sample Size

Giner‐Sorolla

Montoya²,

Aberson³

et al. 2023

Preprint

View full text Add to dashboard Cite

In the wake of the replication crisis, social and personality psychologists have increased attention to power analysis and the adequacy of sample sizes. In this paper, we analyze current controversies in this area, including choosing effect size, why and whether power analyses should be conducted on already-collected data, how to mitigate negative effects of sample size criteria on specific kinds of research, and which power criterion to use. Because our conventions about effect sizes are less clear than some models of inference demand, we advocate that effect size targets be chosen for basis in solid evidence and practical relevance to basic and applied research, rather than on the basis of minimal or field-wide effect sizes. We discuss two alternatives to power analysis, precision analysis and sequential analysis, and end with recommendations for improving the practices of researchers, reviewers, and journal editors in social-personality psychology.

show abstract

Effect size reporting and interpretation in social personality research

Cited by 4 publications

References 42 publications

Determining Negligible Associations in Regression

Determining Negligible Associations in Regression

Power to Detect What? Considerations for Planning and Evaluating Sample Size

Power to Detect What? Considerations for Planning and Evaluating Sample Size

Contact Info

Product

Resources

About