Statistics and the detection of scientific misconduct

Gross, Robert A.

doi:10.1212/wnl.0000000000003390

Cited by 18 publications

(10 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We raised concerns about 33 randomised controlled trial (RCT) reports, ‘affected trial reports’, from one research group in Japan (see online supplementary appendix for list of 33 RCTs) 1 2. Our systematic review published in November 2016 examined these affected trial reports published in the field of osteoporosis over 15 years.…”

Section: Introductionmentioning

confidence: 99%

An investigation into the impact and implications of published papers from retracted research: systematic search of affected literature

et al. 2019

View full text Add to dashboard Cite

ObjectiveAnalyses of the impact of a body of clinical trial reports subject to research misconduct have been few. Our objective was to examine the impact on clinically relevant research of a group of researchers’ trial reports (‘affected trial reports’) affected by research misconduct, and whether identification of misconduct invoked a reappraisal.DesignIn 2016, we used five databases and search engines to identify ‘citing publications’, that is, guidelines, systematic and other reviews, and clinical trials citing any of 12 affected trial reports, published 1998–2011, eventually retracted for research misconduct. The affected trial reports were assessed more likely to have had impact because they had hip fracture outcomes and were in journals with impact factor >4. Two authors assessed whether findings of the citing publications would change if the affected trial reports were removed. In 2018, we searched for evidence that the citing publications had undertaken a reassessment as a result of the potential influence of the affected trial reports.ResultsBy 2016 the affected trial reports were cited in 1158 publications, including 68 systematic reviews, meta-analyses, narrative reviews, guidelines and clinical trials. We judged that 13 guidelines, systematic or other reviews would likely change their findings if the affected trial reports were removed, and in another eight it was unclear if findings would change. By 2018, only one of the 68 citing publications, a systematic review, appeared to have undertaken a reassessment, which led to a correction.ConclusionsWe found evidence that this group of affected trial reports distorted the evidence base. Correction of these distortions is slow, uncoordinated and inconsistent. Unless there is a rapid, systematic, coordinated approach by bibliographic databases, authors, journals and publishers to mitigate the impact of known cases of research misconduct, patients, other researchers and their funders may continue to be adversely affected.

show abstract

Section: Introductionmentioning

confidence: 99%

An investigation into the impact and implications of published papers from retracted research: systematic search of affected literature

et al. 2019

View full text Add to dashboard Cite

show abstract

“…There were several parts to their analysis and the ultimate finding of fraud is not in question as the lead researcher admitted to it and retracted some publications, e.g. Gross [ 2 ]. In this article, we focus on one aspect of the statistical analysis in Bolland et al [ 1 ]: their assessment of the uniformity of the p-values comparing baseline characteristics between randomized treatment groups.…”

Section: Introductionmentioning

confidence: 99%

Correlation among baseline variables yields non-uniformity of p-values

Betensky

Chiou

2017

PLoS ONE

View full text Add to dashboard Cite

A recent paper in Neurology used statistical techniques to investigate the integrity of the randomization in 33 clinical trials conducted by a group of investigators. Without justification, the approach assumed that there would be no impact of correlation among baseline variables. We investigated the impact of correlation on the conclusions of the approach in several large-scale simulation studies that replicated the sample sizes and baseline variables of the clinical trials in question and utilized proper randomization. Additionally, we considered scenarios with larger numbers of baseline variables. We found that, with even moderate correlation, there can be substantial inflation of the type I error of statistical tests of randomization integrity. This is also the case under no correlation, in the presence of some discrete baseline variables, with a large number of variables. Thus, statistical techniques for assessing randomization integrity should be applied with extreme caution given that very low p-values, which are taken as evidence against valid randomization, can arise even in the case of valid randomization, in the presence of correlation. More generally, the use of tests of goodness of fit to uniformity for the purpose of testing a global null hypothesis is not advisable in the presence of correlation.

show abstract

“…Previously, we assessed the distribution of p-values from comparisons between baseline variables in a group of RCTs about which concerns had been raised [3]. To date, at least some of these RCTs were determined to be fraudulent [5]. In theory, because participants in a RCT are randomly allocated to study groups, the expected distribution of p-values from comparisons between randomised groups for independent baseline continuous variables (baseline p-values) is uniform.…”

Section: Introductionmentioning

confidence: 99%