Improving the Dependability of Research in Personality and Social Psychology

Abstract

In this article, the Society for Personality and Social Psychology (SPSP) Task Force on Publication and Research Practices offers a brief statistical primer and recommendations for improving the dependability of research. Recommendations for research practice include (a) describing and addressing the choice of N (sample size) and consequent issues of statistical power, (b) reporting effect sizes and 95% confidence intervals (CIs), (c) avoiding "questionable research practices" that can inflate the probability of Type I error, (d) making available research materials necessary to replicate reported results, (e) adhering to SPSP's data sharing policy, (f) encouraging publication of high-quality replication studies, and (g) maintaining flexibility and openness to alternative standards and methods. Recommendations for educational practice include (a) encouraging a culture of "getting it right," (b) teaching and encouraging transparency of data reporting, (c) improving methodological instruction, and (d) modeling sound science and supporting junior researchers who seek to "get it right."

Solidarity through shared disadvantage: Highlighting shared experiences of discrimination improves relations between stigmatized groups.

“…The psychological research community's best practices are evolving regarding power, sample size, and replication (e.g., Funder et al, 2014;Miller & Ulrich, 2016;Nosek & Bar-Anan, 2012;Simmons, Nelson, & Simonsohn, 2011). When one's study populations are large in number, replenish regularly, and participate for small monetary compensation, it may be relatively straightforward to increase sample size and conduct replications.…”

Psychological science’s replicability crisis and what it means for science in the courtroom

“…These concerns were eventually echoed by an American Psychological Association Task Force (Wilkinson Task Force, 1999) with explicit recommendations for overhauling statistics in psychological research. The very same recommendations are being made once again in reaction to the replicability crisis (Funder et al, 2013).…”

“…Some writers (Funder et al, 2013) trace it to an article by Ioannidis (2005) burgeoning area of research that uses fMRI to study social psychological processes were higher than would statistically be expected given how the measurements were taken and analyses performed. Briefly, the source of the problem appears to be that experimenters naively obtained measurements in a non-independent manner.…”

“…Researchers believe that lack of access to raw data played a part in the replicability crisis (Funder et al, 2013). In other words, statistical analyses could not be vetted by methodologists because the raw data was not available.…”

“…Correspondingly, journals and governing bodies took steps to address the crisis (e.g., Funder et al, 2013). In the subsequent section I will review the sources of the replicability crisis with the Daubert factors as an armature for my analysis.…”