2021
DOI: 10.7554/elife.67995
|View full text |Cite
|
Sign up to set email alerts
|

Challenges for assessing replicability in preclinical cancer biology

Abstract: We conducted the Reproducibility Project: Cancer Biology to investigate the replicability of preclinical research in cancer biology. The initial aim of the project was to repeat 193 experiments from 53 high-impact papers, using an approach in which the experimental protocols and plans for data analysis had to be peer reviewed and accepted for publication before experimental work could begin. However, the various barriers and challenges we encountered while designing and conducting the experiments meant that we… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
2
1
1

Citation Types

0
99
0

Year Published

2021
2021
2023
2023

Publication Types

Select...
6
1

Relationship

1
6

Authors

Journals

citations
Cited by 150 publications
(99 citation statements)
references
References 192 publications
0
99
0
Order By: Relevance
“…If, for example, original experiments were influenced by publication bias, with null results being ignored at greater rates, then the meta-analytic evidence would be biased. The use of preregistration and complete outcome reporting in the project eliminated the possibility of publication bias in the replication experiments ( Errington et al, 2021b ), but it may be present in the original experiments. Nevertheless, we combined the two sets of results by weighting each finding by the inverse of its variance to estimate the effect size and effect precision.…”
Section: Resultsmentioning
confidence: 99%
See 1 more Smart Citation
“…If, for example, original experiments were influenced by publication bias, with null results being ignored at greater rates, then the meta-analytic evidence would be biased. The use of preregistration and complete outcome reporting in the project eliminated the possibility of publication bias in the replication experiments ( Errington et al, 2021b ), but it may be present in the original experiments. Nevertheless, we combined the two sets of results by weighting each finding by the inverse of its variance to estimate the effect size and effect precision.…”
Section: Resultsmentioning
confidence: 99%
“…In the Reproducibility Project: Cancer Biology, we sought to acquire evidence about the replicability of preclinical research in cancer biology by repeating selected experiments from 53 high-impact papers published in 2010, 2011, and 2012 ( Errington et al, 2014 ). We describe in a companion paper ( Errington et al, 2021b ) the challenges we encountered while repeating these experiments. These barriers include: shortcomings in documentation of the original methodology; failures of transparency in original findings and protocols; failures to share original data, reagents, and other materials; methodological challenges encountered during the execution of the replication experiments.…”
Section: Introductionmentioning
confidence: 99%
“…As mentioned previously, according to the Editors' summaries, 2 of the 17 published Replication Studies reported findings that could not be interpreted (due to unexpected challenges in repeating the original experiments), and 33 studies were abandoned due to cost overruns, difficulties in securing research materials, or a lack of cooperation from the original authors. The RPCB team found that the original authors were bimodal in their helpfulness when it came to providing feedback and sharing data and materials: 26% were extremely helpful, but 32% were not at all helpful or did not respond ( Errington et al, 2021c ).…”
Section: Preclinical Studies As Diagnostic Machinesmentioning
confidence: 99%
“…The problems the RPCB team experienced in terms of important information not being included in the original research articles, or the original authors not sharing data and/or reagents ( Errington et al, 2021c ), also points to a diagnostic machine whose workings are often opaque and that leaves a very patchy audit trail for its outputs.…”
Section: Preclinical Studies As Diagnostic Machinesmentioning
confidence: 99%
See 1 more Smart Citation