Subgroup Analysis of Trials Is Rarely Easy (SATIRE): a study protocol for a systematic review to characterize the analysis, reporting, and claim of subgroup effects in randomized trials

Sun, Xin; Briel, Matthias; Busse, Jason W.; Akl, Elie A.; You, John J.; Mejza, Filip; Bała, Małgorzata M; Diaz-Granados, Natalia; Bassler, Dirk; Mertz, Dominik; Srinathan, Sadeesh; Vandvik, Per Olav; Málaga, Germán; Alshurafa, Mohamad; Dahm, Philipp; Alonso‐Coello, Pablo; Heels‐Ansdell, Diane; Bhatnagar, Neera; Johnston, Bradley C.; Wang, Li; Walter, Stephen D.; Altman, Douglas G.; Guyatt, Gordon

doi:10.1186/1745-6215-10-101

Cited by 52 publications

(67 citation statements)

References 28 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on a previous study 23 , we anticipated in our protocol that about 20% of LBP trials would report a subgroup effect and therefore two random samples of 250 papers would be needed to reach our target of 100 studies. However, we screened three random samples of 250 potentially eligible records (750 records) plus 636 records from the updated search one year after the initial search, to accrue 66 trials that reported a subgroup analysis.…”

Section: Resultsmentioning

confidence: 99%

“…Using the definition from a previous study 23 , we considered a subgroup claim to have been made when the investigators stated in the abstract, results or discussion that the effect of intervention differed, or may have differed, according to the status of a subgroup variable. We only included trial reports that claimed a subgroup effect; trials that included a subgroup analysis but did not claim an effect were not included.…”

Section: Types Of Studiesmentioning

confidence: 99%

See 1 more Smart Citation

A systematic review reveals that the credibility of subgroup claims in low back pain trials was low

Saragiotto

Maher

Moseley

et al. 2016

Journal of Clinical Epidemiology

Self Cite

View full text Add to dashboard Cite

Section: Resultsmentioning

confidence: 99%

Section: Types Of Studiesmentioning

confidence: 99%

A systematic review reveals that the credibility of subgroup claims in low back pain trials was low

Saragiotto

Maher

Moseley

et al. 2016

Journal of Clinical Epidemiology

Self Cite

View full text Add to dashboard Cite

“…These are summarized in the Table (inspired from published literature 15,20,31 ). Subgroup effects gain credibility when a small number of plausible groups, perhaps identified in previous studies, has been prespecified in a detailed published trial protocol.…”

Section: What Can Give Support To a Subgroup Finding?mentioning

confidence: 99%

“…[11][12][13][14][15]23 In a clinical trial, it is usual to collect detailed information on patient characteristics and the specific outcome measures. Researchers can perform many separate analyses in the hope that "something will turn up" that has a P value lower than .05.…”

Section: Problems With Subgroup Analysesmentioning

confidence: 99%

“…[10][11][12][13][14] In a large systematic review aiming to study the analysis, reporting, and claims of subgroup effects in a representative sample of recent RCTs, investigators showed that 45% of 139 RCTs reported subgroup effects for any outcome. 15 Advocates of subgroup analyses are alarmed by the risk of missing important differences in treatment effect, which could result in failure to detect important differences in heterogeneous populations. [16][17][18][19] Opponents describe subgroup analyses as "fishing expeditions" and "data dredging exercises."…”

Section: Subgroup Analysis In Randomized Controlled Trials and Observmentioning

confidence: 99%

See 1 more Smart Citation

The Problem of Subgroup Analyses: An Example from a Trial on Ruptured Intracranial Aneurysms

Naggara

Raymond²,

Guilbert³

et al. 2011

AJNR Am J Neuroradiol

View full text Add to dashboard Cite

Evaluation of Evidence of Statistical Support and Corroboration of Subgroup Claims in Randomized Clinical Trials

et al. 2017

View full text Add to dashboard Cite

IMPORTANCE Many published randomized clinical trials (RCTs) make claims for subgroup differences. OBJECTIVE To evaluate how often subgroup claims reported in the abstracts of RCTs are actually supported by statistical evidence (P < .05 from an interaction test) and corroborated by subsequent RCTs and meta-analyses. DATA SOURCES This meta-epidemiological survey examines data sets of trials with at least 1 subgroup claim, including Subgroup Analysis of Trials Is Rarely Easy (SATIRE) articles and Discontinuation of Randomized Trials (DISCO) articles. We used Scopus (updated July 2016) to search for English-language articles citing each of the eligible index articles with at least 1 subgroup finding in the abstract. STUDY SELECTION Articles with a subgroup claim in the abstract with or without evidence of statistical heterogeneity (P < .05 from an interaction test) in the text and articles attempting to corroborate the subgroup findings. DATA EXTRACTION AND SYNTHESIS Study characteristics of trials with at least 1 subgroup claim in the abstract were recorded. Two reviewers extracted the data necessary to calculate subgroup-level effect sizes, standard errors, and the P values for interaction. For individual RCTs and meta-analyses that attempted to corroborate the subgroup findings from the index articles, trial characteristics were extracted. Cochran Q test was used to reevaluate heterogeneity with the data from all available trials. MAIN OUTCOMES AND MEASURES The number of subgroup claims in the abstracts of RCTs, the number of subgroup claims in the abstracts of RCTs with statistical support (subgroup findings), and the number of subgroup findings corroborated by subsequent RCTs and meta-analyses. RESULTS Sixty-four eligible RCTs made a total of 117 subgroup claims in their abstracts. Of these 117 claims, only 46 (39.3%) in 33 articles had evidence of statistically significant heterogeneity from a test for interaction. In addition, out of these 46 subgroup findings, only 16 (34.8%) ensured balance between randomization groups within the subgroups (eg, through stratified randomization), 13 (28.3%) entailed a prespecified subgroup analysis, and 1 (2.2%) was adjusted for multiple testing. Only 5 (10.9%) of the 46 subgroup findings had at least 1 subsequent pure corroboration attempt by a meta-analysis or an RCT. In all 5 cases, the corroboration attempts found no evidence of a statistically significant subgroup effect. In addition, all effect sizes from meta-analyses were attenuated toward the null. CONCLUSIONS AND RELEVANCE A minority of subgroup claims made in the abstracts of RCTs are supported by their own data (ie, a significant interaction effect). For those that have statistical support (P < .05 from an interaction test), most fail to meet other best practices for subgroup tests, including prespecification, stratified randomization, and adjustment for multiple testing. Attempts to corroborate statistically significant subgroup differences are rare; when done, the initially observed subgroup diffe...

show abstract

Subgroup Analysis of Trials Is Rarely Easy (SATIRE): a study protocol for a systematic review to characterize the analysis, reporting, and claim of subgroup effects in randomized trials

Cited by 52 publications

References 28 publications

A systematic review reveals that the credibility of subgroup claims in low back pain trials was low

A systematic review reveals that the credibility of subgroup claims in low back pain trials was low

The Problem of Subgroup Analyses: An Example from a Trial on Ruptured Intracranial Aneurysms

Evaluation of Evidence of Statistical Support and Corroboration of Subgroup Claims in Randomized Clinical Trials

Contact Info

Product

Resources

About