Misuse of Baseline Comparison Tests and Subgroup Analyses in Surgical Trials

Abstract

It is unclear whether the misuse of statistical tests that compare patients' baseline characteristics and subgroup analyses in randomized controlled trials can be extrapolated to the surgical literature. We did an observational study evaluating the current use of baseline comparability tests and subgroup analyses in surgical randomized controlled trials. Published surgical randomized controlled trials in four medical journals were identified. We also identified randomized controlled trials in the Journal of Bone and Joint Surgery (American and British volumes). We identified 72 randomized controlled trials, with a mean of 10 +/- 8 baseline variables. Of 166 significance tests, 17 (10%) were significant. Twenty-seven (38%) trials included 54 subgroup analyses with a minimum of one and maximum of 23 subgroup analyses per study. Inappropriate emphasis on subgroup analyses occurred frequently. Forty-nine (91%) analyses were performed post hoc without prior hypotheses. Investigators reported differences between subgroups in 31 (57%) of the analyses, all of which were featured in the summary or conclusion. These inferences may be misleading, making their application to clinical practice unwarranted.

Evidence for the Selective Reporting of Analyses and Discrepancies in Clinical Trials: A Systematic Review of Cohort Studies of Clinical Trials

“…The fourth study found subgroup analyses reported in only seven RCTs, and no details had been included in the trial registry entries for six of these RCTs, while the seventh had no discrepancies [6]. In seven studies [11],[13],[18],[21],[22],[26],[30], where the comparison was mostly made between the methods and results sections of the trial publication, it was found that a number of subgroup analyses conducted were not pre-specified (range: 14% [8/58] to 91% [49/54] of RCTs), pre-specified, but not reported in the publication (range: 27% [3/11] to 53% [9/17]), or contained a mixture of pre-specified and non-pre-specified subgroup analyses (range: 10% [6/58] to 65% [135/207]).…”

Credibility of claims of subgroup effects in randomised controlled trials: systematic review

“…The findings of subgroup analyses offer the promise of individualising patient care, and such analyses are common in randomised controlled trials, with 40-65% reporting them. [1][2][3][4][5][6][7] In particular, claims of subgroup effect, in which the authors convey a conviction or belief of a difference in treatment effects between patient subgroups, can have a substantial impact on clinical practice and policy decision. One study found that in 35 randomised controlled trials published in top general medical journals reporting subgroup analyses, 21 (60%) claimed subgroup effects.…”

Is a Subgroup Claim Believable?

“…Such secondary subgroup analyses are common in randomized trials. A recent survey found that 38% (twenty-seven) of seventy-two surgical trials included subgroup analyses, and 57.4% (thirty-one) of fifty-four reported subgroup analyses claimed subgroup effects (that is, that the effect differed across subgroup categories) 3 . On the one hand, the real subgroup effects are important and informative, which allows targeting use of therapies to individual patients to achieve optimal treatments.…”