ObjectivesPrespecified progression criteria can inform the decision to progress from an external randomised pilot trial to a definitive randomised controlled trial. We assessed the characteristics of progression criteria reported in external randomised pilot trial protocols and results publications, including whether progression criteria were specified a priori and mentioned in prepublication peer reviewer reports.Study designMethodological review.MethodsWe searched four journals through PubMed: British Medical Journal Open, Pilot and Feasibility Studies, Trials and Public Library of Science One. Eligible publications reported external randomised pilot trial protocols or results, were published between January 2018 and December 2019 and reported progression criteria. We double data extracted 25% of the included publications. Here we report the progression criteria characteristics.ResultsWe included 160 publications (123 protocols and 37 completed trials). Recruitment and retention were the most frequent indicators contributing to progression criteria. Progression criteria were mostly reported as distinct thresholds (eg, achieving a specific target; 133/160, 83%). Less than a third of the planned and completed pilot trials that included qualitative research reported how these findings would contribute towards progression criteria (34/108, 31%). The publications seldom stated who established the progression criteria (12/160, 7.5%) or provided rationale or justification for progression criteria (44/160, 28%). Most completed pilot trials reported the intention to proceed to a definitive trial (30/37, 81%), but less than half strictly met all of their progression criteria (17/37, 46%). Prepublication peer reviewer reports were available for 153/160 publications (96%). Peer reviewer reports for 86/153 (56%) publications mentioned progression criteria, with peer reviewers of 35 publications commenting that progression criteria appeared not to be specified.ConclusionsMany external randomised pilot trial publications did not adequately report or propose prespecified progression criteria to inform whether to proceed to a future definitive randomised controlled trial.
Background
Randomised controlled trials (RCTs) need to be reported so that their results can be unambiguously and robustly interpreted. Binary outcomes yield unique challenges, as different analytical approaches may produce relative, absolute, or no treatment effects, and results may be particularly sensitive to the assumptions made about missing data. This review of recently published RCTs aimed to identify the methods used to analyse binary primary outcomes, how missing data were handled, and how the results were reported.
Methods
Systematic review of reports of RCTs published in January 2019 that included a binary primary outcome measure. We identified potentially eligible English language papers on PubMed, without restricting by journal or medical research area. Papers reporting the results from individually randomised, parallel-group RCTs were included.
Results
Two hundred reports of RCTs were included in this review. We found that 64% of the 200 reports used a chi-squared-style test as their primary analytical method. Fifty-five per cent (95% confidence interval 48% to 62%) reported at least one treatment effect measure, and 38% presented only a p value without any treatment effect measure. Missing data were not always adequately described and were most commonly handled using available case analysis (69%) in the 140 studies that reported missing data. Imputation and best/worst-case scenarios were used in 21% of studies. Twelve per cent of articles reported an appropriate sensitivity analysis for missing data.
Conclusions
The statistical analysis and reporting of treatment effects in reports of randomised trials with a binary primary endpoint requires substantial improvement. Only around half of the studied reports presented a treatment effect measure, hindering the understanding and dissemination of the findings. We also found that published trials often did not clearly describe missing data or sensitivity analyses for these missing data. Practice for secondary endpoints or observational studies may differ.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.