Trends and predictors of biomedical research quality, 1990–2015: a meta-research study

Abstract

ObjectiveTo measure the frequency of adequate methods, inadequate methods and poor reporting in published randomised controlled trials (RCTs) and test potential factors associated with adequacy of methods and reporting.DesignRetrospective analysis of RCTs included in Cochrane reviews. Time series describes the proportion of RCTs using adequate methods, inadequate methods and poor reporting. A multinomial logit model tests potential factors associated with methods and reporting, including funding source, first author affiliation, clinical trial registration status, study novelty, team characteristics, technology and geography.DataRisk of bias assessments for random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data and selective reporting, for each RCT, were mapped to bibliometric and funding data.OutcomesRisk of bias on six methodological dimensions and RCT-level overall assessment of adequate methods, inadequate methods or poor reporting.ResultsThis study analysed 20 571 RCTs. 5.7% of RCTs used adequate methods (N=1173). 59.3% used inadequate methods (N=12 190) and 35.0% were poorly reported (N=7208). The proportion of poorly reported RCTs decreased from 42.5% in 1990 to 30.2% in 2015. The proportion of RCTs using adequate methods increased from 2.6% in 1990 to 10.3% in 2015. The proportion of RCTs using inadequate methods increased from 54.9% in 1990 to 59.5% in 2015. Industry funding, top pharmaceutical company affiliation, trial registration, larger authorship teams, international teams and drug trials were associated with a greater likelihood of using adequate methods. National Institutes of Health funding and university prestige were not.ConclusionEven though reporting has improved since 1990, the proportion of RCTs using inadequate methods is high (59.3%) and increasing, potentially slowing progress and contributing to the reproducibility crisis. Stronger incentives for the use of adequate methods are needed.

Comparison between 2000 and 2018 on the reporting of statistical significance and clinical relevance in physiotherapy clinical trials in six major physiotherapy journals: a meta-research design

“…Comparison with other studies A previous study evaluating overall quality of methods in biomedical RCTs, including randomisation, blinding and selective reporting, concluded that 59.3% of RCTs used inadequate methods (meaning scoring high risk of bias on one or more of the six Cochrane risk of bias items) and 35% of RCTs were poorly reported (meaning providing not enough information in the methods to decide on adequate or inadequate methods). 28 Comparable findings have been found in physiotherapy RCTs in the PEDro database 23 and evaluation of manual therapy trials. 29 30 While reporting of effect estimates in our selection of high-quality physiotherapy literature differs between 2000 and 2018, still most papers did not adhere to the reporting recommendations provided by the ASA and CONSORT-statements with regards to statistical significance testing and reliance on p values to interpret results.…”

Using bibliometrics to evaluate outcomes and influence of translational biomedical research centers

“…Specifically, an average of three to four-person teams produced each multi-authored paper. Existing SciTS literature has stated that co-authors are more heavily cited than single-authors, an increase in authors leads to increased research impact, and increasing team size predicts research quality increases (4,5,37).…”

“…Specifically, an average of three to four-person teams produced each multi-authored paper. Existing SciTS literature has stated that co-authors are more heavily cited than single authors, an increase in authors leads to increased research impact, and increasing team size predicts research quality increases [4,5,37]. The average CNTN research team size is equal to the observed average team size across the landscape of science, engineering, and social sciences [5].…”

Blinding in Clinical Trials: Seeing the Big Picture

“…Despite exponential growth in the number of clinical trials conducted yearly over the past two decades [ 5 ], multiple authors contend that the methodological quality of clinical trials has remained stagnant or even declined [ 6 , 7 ], such that true practice-guiding evidence on a broad range of medial topics paradoxically lags behind [ 8 , 9 , 10 ]. Blinding is one aspect of clinical trial design that remains particularly underutilized—although this methodological feature is not universally attainable, blinding is still implemented in only a fraction of clinical trials in which it is, in fact, deemed feasible [ 11 , 12 , 13 ].…”