Replications in Psychology Research

Abstract

Recent controversies in psychology have spurred conversations about the nature and quality of psychological research. One topic receiving substantial attention is the role of replication in psychological science. Using the complete publication history of the 100 psychology journals with the highest 5-year impact factors, the current article provides an overview of replications in psychological research since 1900. This investigation revealed that roughly 1.6% of all psychology publications used the term replication in text. A more thorough analysis of 500 randomly selected articles revealed that only 68% of articles using the term replication were actual replications, resulting in an overall replication rate of 1.07%. Contrary to previous findings in other fields, this study found that the majority of replications in psychology journals reported similar findings to their original studies (i.e., they were successful replications). However, replications were significantly less likely to be successful when there was no overlap in authorship between the original and replicating articles. Moreover, despite numerous systemic biases, the rate at which replications are being published has increased in recent decades.

Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education

“…Consistent with findings from other fields (see Makel et al, 2012;Makel & Plucker, 2014), Makel et al (2016) and Lemons et al (2016) reported that only 0.5% and 0.4%, respectively, of articles published in special education were replication studies; and most of those were conceptual replications, which do not directly test the validity of previous findings. Although other reviews (Banerjee, Movahedazarhouligh, Millen, & Luckner, 2018;Cook, Collins, Cook, & Cook, 2016;PROMOTING OPEN SCIENCE 11 Therrien, Matthews, Hirsch, & Solis, 2016) indicated that special education researchers often conduct studies that conceptually replicate at least some aspects of previous studies, these authors found that special education researchers (a) seldom explicitly identify the purpose of their research as replication, and (b) almost never conduct direct replications.…”

“…Unfortunately, replication studies are seldom conducted (Makel, Plucker, & Hegarty, 2012;Makel & Plucker, 2014). Interestingly, when researchers have conducted replication studies, results have often failed to validate original findings (Camerer et al, 2016, in press;Ebersole et al, 2016;Klein et al, 2014;Open Science Collaboration, 2015).…”

When null hypothesis significance testing is unsuitable for research: a reassessment

“…In contrast to the above, until recently replication studies have been very rare in many scientific fields; lack of replication efforts has been a particular problem in the psychological sciences (Makel et al, 2012), but this may hopefully change now with the wide attention that replication has received (Aarts et al, 2015).…”

Can the behavioral sciences self-correct? A social epistemic study

“…In psychology most replications are conceptual and not direct. Makel et al (2012) systematically analyzed articles in the top 100 psychology journals since 1900, and they found that only 1.07% of them reported replication attempts. And even worse, only 14% of all replications were direct replications.…”