Two years into the Brazilian Reproducibility Initiative: reflections on conducting a large-scale replication of Brazilian biomedical science

Abstract

Scientists have increasingly recognised that low methodological and analytical rigour combined with publish-or-perish incentives can make the published scientific literature unreliable. As a response to this, large-scale systematic replications of the literature have emerged as a way to assess the problem empirically. The Brazilian Reproducibility Initiative is one such effort, aimed at estimating the reproducibility of Brazilian biomedical research. Its goal is to perform multicentre replications of a quasi-random sample of at least 60 experiments from Brazilian articles published over a 20-year period, using a set of common laboratory methods. In this article, we describe the challenges of managing a multicentre project with collaborating teams across the country, as well as its successes and failures over the first two years. We end with a brief discussion of the Initiative’s current status and its possible future contributions after the project is concluded in 2021.

Estimating the replicability of highly cited clinical research (2004-2018)

“…Lastly, we relied on replications that were published in the literature. As the existence and publication of these replications are subject both to the interest of researchers to perform them and to that of editors and reviewers to publish them, the approach in our study is not directly comparable to the systematic replication attempts that have been performed in other areas of the literature (1)(2)(3)(4)(5)(6)(7)(8). This is particularly important given that the majority of highly cited studies in our sample had no available replications according to our criteria -thus, selectiveness in performing or publishing replications may have biased our replicability rates.…”

“…The replicability of published research has been recently questioned in different scientific fields, with replication rates shown to be variable and often low (1)(2)(3)(4)(5)(6)(7)(8). Whether this represents a "reproducibility crisis" is open to debate (9), as defining what constitutes a successful replication is not trivial (10).…”

Research on scalable graphene faces a reproducibility gap

“…It was founded by, among others, Mendeley and PLOS. The Brazilian Reproducibility Initiative goes the extra mile by repeating between 60 and 100 published experiments, each in three different laboratories throughout the country 8 , 9 . A number of initiatives like this exist across a wide variety of research fields; perhaps the most famous is the 8-year Reproducibility Project: Cancer Biology (RPCB), which reported that half of top cancer studies failed to be replicated 10 .…”

O aumento da má conduta científica nas pesquisas da área de saúde: impressão ou realidade?

“…30 Criada também em 2018, a Iniciativa Brasileira de Reprodutibilidade se configura como um grupo que, motivados pelos indícios de baixa validade interna das publicações científicas e a forte cultura do publique ou pereça, busca quantificar a reprodutibilidade da ciência biomédica brasileira e portanto, quanto dos seus resultados são confiáveis. 31,32 Já no ano de 2019, a produção científica de má qualidade foi pauta da 6ª Conferência Mundial sobre Integridade Científica em Hong Kong, encontro no qual foram levantados diversos questionamentos acerca do comportamento dos pesquisadores, no intuito de repensar a forma como a produção científica é recompensada. 33 A existência desses órgãos em si não indica a magnitude e a extensão das más práticas científicas na área biomédica, mas alerta para um problema grave e com potenciais impactos devastadores.…”