Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses

Sun, Xin; Briel, Matthias; Walter, Stephen D.; Guyatt, Gordon

doi:10.1136/bmj.c117

Cited by 594 publications

(538 citation statements)

References 34 publications

Supporting

Mentioning

513

Contrasting

Unclassified

Order By: Relevance

“…The subgroup effect and its direction was specified a priori, the difference between subgroups was relatively large (about 21 hours), and chance seems an unlikely explanation (P<0.001). Credibility of the effect, however, is undermined34 as the effect modification is suggested by comparison between rather than within studies, and we found no similar difference in any other outcome. In addition, the only randomised controlled trial that compared oral and intramuscular treatment with dexamethasone reported no significant difference in any outcome 25…”

Section: Discussioncontrasting

confidence: 57%

Corticosteroids for treatment of sore throat: systematic review and meta-analysis of randomised trials

Sadeghirad

Siemieniuk

Brignardello‐Petersen

et al. 2017

BMJ

View full text Add to dashboard Cite

Objective To estimate the benefits and harms of using corticosteroids as an adjunct treatment for sore throat. Design Systematic review and meta-analysis of randomised control trials. Data sources Medline, Embase, Cochrane Central Register of Controlled Trials (CENTRAL), trial registries up to May 2017, reference lists of eligible trials, related reviews. Study selection Randomised controlled trials of the addition of corticosteroids to standard clinical care for patients aged 5 or older in emergency department and primary care settings with clinical signs of acute tonsillitis, pharyngitis, or the clinical syndrome of sore throat. Trials were included irrespective of language or publication status. Review methods Reviewers identified studies, extracted data, and assessed the quality of the evidence, independently and in duplicate. A parallel guideline committee (BMJ Rapid Recommendation) provided input on the design and interpretation of the systematic review, including the selection of outcomes important to patients. Random effects model was used for meta-analyses. Quality of evidence was assessed with the GRADE approach. Results 10 eligible trials enrolled 1426 individuals. Patients who received single low dose corticosteroids (the most common intervention was oral dexamethasone with a maximum dose of 10 mg) were twice as likely to experience pain relief after 24 hours (relative risk 2.2, 95% confidence interval 1.2 to 4.3; risk difference 12.4%; moderate quality evidence) and 1.5 times more likely to have no pain at 48 hours (1.5, 1.3 to 1.8; risk difference 18.3%; high quality). The mean time to onset of pain relief in patients treated with corticosteroids was 4.8 hours earlier (95% confidence interval −1.9 to −7.8; moderate quality) and the mean time to complete resolution of pain was 11.1 hours earlier (−0.4 to −21.8; low quality) than in those treated with placebo. The absolute pain reduction at 24 hours (visual analogue scale 0-10) was greater in patients treated with corticosteroids (mean difference 1.3, 95% confidence interval 0.7 to 1.9; moderate quality). Nine of the 10 trials sought information regarding adverse events. Six studies reported no adverse effects, and three studies reported few adverse events, which were mostly complications related to disease, with a similar incidence in both groups. Conclusion Single low dose corticosteroids can provide pain relief in patients with sore throat, with no increase in serious adverse effects. Included trials did not assess the potential risks of larger cumulative doses in patients with recurrent episodes of acute sore throat. Systematic review registration PROSPERO CRD42017067808.

show abstract

Section: Discussioncontrasting

confidence: 57%

Corticosteroids for treatment of sore throat: systematic review and meta-analysis of randomised trials

Sadeghirad

Siemieniuk

Brignardello‐Petersen

et al. 2017

BMJ

View full text Add to dashboard Cite

show abstract

“…Furthermore, even when the analysis of a particular factor produces a β W that is statistically and clinical important, there may still be debate about whether the factor is a genuine causal modifier of treatment effect. Sun et al 58. provide guidance for identifying whether differences in subgroups are believable, and this includes consideration of biological plausibility.…”

Section: Discussionmentioning

confidence: 99%

One-stage individual participant data meta-analysis models: estimation of treatment-covariate interactions must avoid ecological bias by separating out within-trial and across-trial information

et al. 2016

View full text Add to dashboard Cite

Stratified medicine utilizes individual‐level covariates that are associated with a differential treatment effect, also known as treatment‐covariate interactions. When multiple trials are available, meta‐analysis is used to help detect true treatment‐covariate interactions by combining their data. Meta‐regression of trial‐level information is prone to low power and ecological bias, and therefore, individual participant data (IPD) meta‐analyses are preferable to examine interactions utilizing individual‐level information. However, one‐stage IPD models are often wrongly specified, such that interactions are based on amalgamating within‐ and across‐trial information. We compare, through simulations and an applied example, fixed‐effect and random‐effects models for a one‐stage IPD meta‐analysis of time‐to‐event data where the goal is to estimate a treatment‐covariate interaction. We show that it is crucial to centre patient‐level covariates by their mean value in each trial, in order to separate out within‐trial and across‐trial information. Otherwise, bias and coverage of interaction estimates may be adversely affected, leading to potentially erroneous conclusions driven by ecological bias. We revisit an IPD meta‐analysis of five epilepsy trials and examine age as a treatment effect modifier. The interaction is −0.011 (95% CI: −0.019 to −0.003; p = 0.004), and thus highly significant, when amalgamating within‐trial and across‐trial information. However, when separating within‐trial from across‐trial information, the interaction is −0.007 (95% CI: −0.019 to 0.005; p = 0.22), and thus its magnitude and statistical significance are greatly reduced. We recommend that meta‐analysts should only use within‐trial information to examine individual predictors of treatment effect and that one‐stage IPD models should separate within‐trial from across‐trial information to avoid ecological bias. © 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

show abstract

“…We believe that a uniform approach4, 21 and greater degree of rigour may be required in assessing the value of these studies prior to publication, incorporation into clinical practice guidelines, and by extension, clinical practice itself.…”

Section: Concluding Thoughtsmentioning

confidence: 99%

Subgroup analyses of randomized clinical trials in heart failure: facts and numbers

et al. 2016

View full text Add to dashboard Cite

Subgroup analyses of major randomized clinical trials in heart failure are published frequently, but their impact on medical knowledge and practice guidelines has not been previously reported. In a novel analysis, we determined number of citations, impact factors, number of authors, and citations in guidelines of both parent trials and sub‐studies; we also qualitatively assessed whether the analyses were described as post‐hoc and non‐pre‐specified. A total of 229 sub‐studies evaluating outcomes in patient subgroups were published (median 6, range 0–36 per trial). The number of subjects in the parent trials positively correlated with number of sub‐studies (rho = 0.51, P = 0.009). The subgroups are frequently not pre‐specified. The impact factors of sub‐studies were lower in comparison to the parent trials as were the number of citations two years after the publication date; in addition, parent trials were cited more frequently in European and American professional guidelines compared with the sub‐studies. We maintain that the sub‐studies derived from major heart failure trials are frequently published, but their contribution to clinical guidelines and medical knowledge are highly debatable.

show abstract

Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses

Cited by 594 publications

References 34 publications

Corticosteroids for treatment of sore throat: systematic review and meta-analysis of randomised trials

Corticosteroids for treatment of sore throat: systematic review and meta-analysis of randomised trials

One-stage individual participant data meta-analysis models: estimation of treatment-covariate interactions must avoid ecological bias by separating out within-trial and across-trial information

Subgroup analyses of randomized clinical trials in heart failure: facts and numbers

Contact Info

Product

Resources

About