Current practice in analysing and reporting binary outcome data—a review of randomised controlled trial reports

Abstract

Background Randomised controlled trials (RCTs) need to be reported so that their results can be unambiguously and robustly interpreted. Binary outcomes yield unique challenges, as different analytical approaches may produce relative, absolute, or no treatment effects, and results may be particularly sensitive to the assumptions made about missing data. This review of recently published RCTs aimed to identify the methods used to analyse binary primary outcomes, how missing data were handled, and how the results were reported. Methods Systematic review of reports of RCTs published in January 2019 that included a binary primary outcome measure. We identified potentially eligible English language papers on PubMed, without restricting by journal or medical research area. Papers reporting the results from individually randomised, parallel-group RCTs were included. Results Two hundred reports of RCTs were included in this review. We found that 64% of the 200 reports used a chi-squared-style test as their primary analytical method. Fifty-five per cent (95% confidence interval 48% to 62%) reported at least one treatment effect measure, and 38% presented only a p value without any treatment effect measure. Missing data were not always adequately described and were most commonly handled using available case analysis (69%) in the 140 studies that reported missing data. Imputation and best/worst-case scenarios were used in 21% of studies. Twelve per cent of articles reported an appropriate sensitivity analysis for missing data. Conclusions The statistical analysis and reporting of treatment effects in reports of randomised trials with a binary primary endpoint requires substantial improvement. Only around half of the studied reports presented a treatment effect measure, hindering the understanding and dissemination of the findings. We also found that published trials often did not clearly describe missing data or sensitivity analyses for these missing data. Practice for secondary endpoints or observational studies may differ.

Caregiving burnout of community-dwelling people with dementia in Hong Kong and New Zealand: a cross-sectional study

“…In this secondary data analysis, the reporting of the binary outcome is confined by the data type collected in the pre-defined assessment tool. In order to remedy the shortcomings in reporting binary outcomes, Rombach and colleagues [ 60 ] recommended medical researchers to use statistical methods that could quantify the confidence intervals of the primary binary outcomes whenever available (including logistic regression in our study), instead of choosing analysis method that could only produce a corresponding p -value (such as chi-square test).…”

Sensitivity to missing not at random dropout in clinical trials: use and interpretation of the Trimmed Means Estimator

“…Six review articles describing the handling of missing data in RCTs found that only 22% of the 649 trials reviewed reported some kind of sensitivity analysis. [13][14][15][16][17][18] The majority retained the missingness assumptions of the original analysis, with only a small subset relaxing the original assumptions, and almost none considering a MNAR mechanism. 13 Applying sensitivity analyses can be limited by the complexity of the methods, and a lack of transparency about the assumptions underlying each method.…”

“…In the broader context of selection models in general, Copas and Jackson 7 defined a bias limit, which is equivalent to the maximum CCA estimator bias that we derive here (15). We extend Copas and Jackson's bias limit to the TM estimator, and define the maximum possible TM bias, which, for the case of lower value trimming, occurs under highest value dropout.…”

Analyses of repeatedly measured continuous outcomes in randomized controlled trials needed substantial improvements

“…and M.Y.) trained in medical statistics with reference to Consolidated Standards of Reporting Trials (CONSORT) [17] and the other methodological studies [18][19][20][21]. Subsequently, they discussed with two experienced methodologists (L.L.…”