Making the scientific literature fail-safe

Abstract

Critical systems are designed to be failsafe. This does not mean that failures cannot occur, but rather that redundant and compensatory mechanisms are engineered into the system to detect and mitigate failures when they occur. The scientific literature is the critical system by which scientific findings are communicated and archived for subsequent reference and analysis. Hence, the reliability of the scientific literature is of the utmost importance to society. However, in recent years, rising numbers of retracted articles, reproducibility problems, and inappropriately duplicated images have increased concern that the scientific literature is unreliable (2-5). Contributing factors may include competition, sloppiness, prioritization of impact over rigor, poor experimental design, inappropriate statistical analysis, and lax ethical standards (6, 7). Although the number of questionable publications represents a very small percentage of the total literature, even a few problematic publications can reduce the credibility of science. Hence, it is important to redouble efforts to improve the reliability of scientific publications. We suggest a seven-point approach to reengineering the scientific literature so that it is better able to prevent and correct its failures. i. Improving graduate and postgraduate training. Training is the foundation of all scientific endeavors. Contemporary graduate scientific training is designed to prepare trainees to perform deep investigation into a highly specialized area (8) but does not necessarily provide students with a broad scientific background. Students are taught in a guild-like environment by a mentor who may or may not

Ethical Consistency and Experience: An Attempt to Influence Researcher Attitudes Toward Questionable Research Practices Through Reading Prompts

“…Various recommendations for improving the integrity and replicability of science have been made or are currently being implemented. These suggestions range from ways of changing researcher behavior, such as improved training (Casadevall & Fang, 2018; Munafò et al, 2017) and motivational badges (Kidwell et al, 2016), to macro-level reforms. These broader reforms include institutional rules for archiving data, lab journals, and scripts for data analysis (Bouter, 2015), enhanced policies (3rd World Conference on Research Integrity, 2013; World Medical Association, 2013), reporting guidelines (Fanelli, 2013), more rigorous journal practices (Vazire, 2016), and modifications to traditional modes of peer review (Herron, 2012).…”

Littérature médicale et COVID-19 : comment trois articles ont influencé les médias et la décision publique en France

“…Ils ne doivent pas accorder trop de crédit immédiat aux articles publiés, car chaque étude en recherche clinique n’a pas pour objectif d’atteindre une vérité mais d’accumuler des preuves pour aller vers un consensus. S’il n’existe pas de solution miracle pour une publication sans faille, nous pensons qu’une meilleure information des scientifiques aux nouveaux modes de publication, un renforcement des procédures éditoriales et d’évaluation, ainsi qu’un développement de la recherche sur la publication, pourraient contribuer à limiter ces failles [50] .…”

A toolkit for open and pluralistic conservation science

“…Indeed, researchers voluntarily “admit” to such practices as continuing data collection after checking if the results have reached a threshold for statistical significance, reporting a set of statistical models as the complete tested set when other candidate models were also tested, or changing to a different statistical analysis after the initial analysis failed to reach statistical significance (Fraser et al., 2018). Worryingly, such practices are often passed down inadvertently from teacher to student (Casadevall & Fang, 2018).…”