PsychDisclosure.org

LeBel, Etienne P.; Borsboom, Denny; Giner‐Sorolla, Roger; Hasselman, Fred; Peters, Kurt R.; Ratliff, Kate A.; Smith, Colin Tucker

doi:10.1177/1745691613491437

Cited by 91 publications

(57 citation statements)

References 34 publications

(42 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This becomes even more problematic, if only the analyses that "worked" are reported and presented as if they were planned from the start (Kerr, 1998;Wagenmakers, Wetzels, Borsboom, Maas, & Kievit, 2012). Survey results show that many psychologists admit to such "questionable research practices" (QRPs; Agnoli, Wicherts, Veldkamp, Albiero, & Cubelli, 2017;John et al, 2012), and use of study registers and later disclosures by researchers provide direct evidence that indeed some of these practices are quite common (Franco, Malhotra, & Simonovits, 2016;LeBel et al, 2013).…”

Section: Bias and Errorsmentioning

confidence: 98%

“…These problems occur even when all other research practices are ideal, and there is strong evidence that they are not. Researchers have a strong focus on reporting significant results (Franco et al, 2014;LeBel et al, 2013). To obtain significant results they seem to make strategic use of flexibility in data analysis, also referred to as "researcher degrees of freedom" (Agnoli et al, 2017;John et al, 2012;Simmons et al, 2011;but see Fiedler & Schwarz, 2016).…”

Section: Power In Intelligence Researchmentioning

confidence: 99%

See 1 more Smart Citation

Research on Research: A Meta-Scientific Study of Problems and Solutions in Psychological Science

Nuijten¹

2018

Preprint

View full text Add to dashboard Cite

SUMMARY DOCTORAL DISSERTATION: Psychology is facing a “replication crisis”. Many psychological findings could not be replicated in novel samples, which lead to the growing concern that many published findings are overly optimistic or even false. In this dissertation, we investigated potential indicators of problems in the published psychological literature. In Part I of this dissertation, we looked at inconsistencies in reported statistical results in published psychology papers. To facilitate our research, we developed the free tool statcheck; a “spellchecker” for statistics. In Part II, we investigated bias in published effect sizes. We showed that in the presence of publication bias, the overestimation of effects can become worse if you combine studies. Indeed, in meta-analyses from the social sciences we found strong evidence that published effects are overestimated. These are worrying findings, and it is important to think about concrete solutions to improve the quality of psychological research. Some of the solutions we propose are preregistration, replication, and transparency. We argue that to select the best strategies to improve psychological science, we need research on research: meta-research.

show abstract

Section: Bias and Errorsmentioning

confidence: 98%

Section: Power In Intelligence Researchmentioning

confidence: 99%

Research on Research: A Meta-Scientific Study of Problems and Solutions in Psychological Science

Nuijten¹

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…For instance, higher reporting standards have recently been instituted at several prominent psychology journals [21,87,88,47]. At such journals (e.g., Psychological Science, Memory & Cognition, Attention,…”

Section: New Initiatives and Reformsmentioning

confidence: 99%

A New Replication Norm for Psychology

LeBel

2015

Collabra

Self Cite

View full text Add to dashboard Cite

There is growing consensus that psychology has a replicability "crisis of confidence" [70,69,89,66,67], stemming from the fact that a growing number of findings cannot be replicated via high-powered independent replication attempts that duplicate the original methodology as closely as possible. Across all areas of psychology, there is a growing list of (prominent) findings that have not held up to independent replication attempts, including findings from cognitive psychology (retrieval- [38,17,51,18,31,57,72,71,94,44,46,52,34]).More generalizable evidence supporting a general replicability problem comes from an ambitious and unprecedented large-scale crowdsourced project, the Reproducibility Project [66,67]. In this project, researchers were unable to replicate about 60% (out of 100) of findings from the 2008 issues of Psychological Science, Journal of Personality and Social Psychology, and Journal of Experimental Psychology: Learning, Memory, and Cognition [68]. In another large-scale meta-scientific investigation, about 70% (16 out of 23) of important findings from cognitive and social psychology could also not be replicated [64]. Though there are different ways to interpret successful versus unsuccessful replication results [91], taken together, these observations strongly suggest psychology currently has a general replicability problem (as do several other areas of science including cancer cell biology and cardiovascular health literatures [8,75]). New Initiatives and ReformsSeveral new initiatives have been launched to improve research practices in order to increase the reliability of findings in psychology. For instance, higher reporting standards have recently been instituted at several prominent psychology journals [21,87,88,47]. At such journals (e.g., Psychological Science, Memory & Cognition, Attention, ORIGINAL RESEARCH REPORT A New Replication Norm for PsychologyEtienne P. LeBel * In recent years, there has been a growing concern regarding the replicability of findings in psychology, including a mounting number of prominent findings that have failed to replicate via high-powered independent replication attempts. In the face of this replicability "crisis of confidence", several initiatives have been implemented to increase the reliability of empirical findings. In the current article, I propose a new replication norm that aims to further boost the dependability of findings in psychology. Paralleling the extant social norm that researchers should peer review about three times as many articles that they themselves publish per year, the new replication norm states that researchers should aim to independently replicate important findings in their own research areas in proportion to the number of original studies they themselves publish per year (e.g., a 4:1 original-to-replication studies ratio). I argue this simple approach could significantly advance our science by increasing the reliability and cumulative nature of our empirical knowledge base, accelerating our theoretical understanding of psycholo...

show abstract

“…[1][2][3][4][5]). We have also seen a rise 3 in the number of government and private funders mandating open access and open data [6,7], and the 4 emergence of the Open Research Funders Group (www.orfg.org).…”

mentioning

confidence: 99%

“…We have also seen a rise 3 in the number of government and private funders mandating open access and open data [6,7], and the 4 emergence of the Open Research Funders Group (www.orfg.org). These initiatives have been key in raising 5 awareness and acceptance of open scholarship. However, despite these advances, I believe we have hit a 6 wall that is impeding widespread adoption.…”

mentioning

confidence: 99%

Imagining the ‘open’ university: Sharing science to improve research and education

McKiernan¹

2017

Preprint

View full text Add to dashboard Cite

Open scholarship, such as the sharing of articles, code, data, and educational resources, has the potential to improve university research and education, as well as increase the impact universities can have beyond their own walls. To support this perspective, I present evidence from case studies, published literature, and personal experiences as a practicing open scholar. I describe some of the challenges inherent to practicing open scholarship, and some of the tensions created by incompatibilities between institutional policies and personal practice. To address this, I propose several concrete actions universities could take to support open scholarship, and outline ways in which such initiatives could benefit the public as well as institutions. Importantly, I do not think most of these actions would require new funding, but rather a redistribution of existing funds and a rewriting of internal policies to be er align with university missions of knowledge dissemination and societal impact.

show abstract

PsychDisclosure.org

Cited by 91 publications

References 34 publications

Research on Research: A Meta-Scientific Study of Problems and Solutions in Psychological Science

Research on Research: A Meta-Scientific Study of Problems and Solutions in Psychological Science

A New Replication Norm for Psychology

Imagining the ‘open’ university: Sharing science to improve research and education

Contact Info

Product

Resources

About