Estimands in published protocols of randomised trials: urgent improvement needed

Rehal

et al. 2022

BMJ

Self Cite

Objectives To evaluate how often the precise research question being addressed about an intervention (the estimand) is stated or can be determined from reported methods, and to identify what types of questions are being investigated in phase 2-4 randomised trials. Design Systematic review of the clarity of research questions being investigated in randomised trials in 2020 in six leading general medical journals. Data source PubMed search in February 2021. Eligibility criteria for selecting studies Phase 2-4 randomised trials, with no restrictions on medical conditions or interventions. Cluster randomised, crossover, non-inferiority, and equivalence trials were excluded. Main outcome measures Number of trials that stated the precise primary question being addressed about an intervention (ie, the primary estimand), or for which the primary estimand could be determined unambiguously from the reported methods using statistical knowledge. Strategies used to handle post-randomisation events that affect the interpretation or existence of patient outcomes, such as intervention discontinuations or uses of additional drug treatments (known as intercurrent events), and the corresponding types of questions being investigated. Results 255 eligible randomised trials were identified. No trials clearly stated all the attributes of the estimand. In 117 (46%) of 255 trials, the primary estimand could be determined from the reported methods. Intercurrent events were reported in 242 (95%) of 255 trials; but the handling of these could only be determined in 125 (49%) of 255 trials. Most trials that provided this information considered the occurrence of intercurrent events as irrelevant in the calculation of the treatment effect and assessed the effect of the intervention regardless (96/125, 77%)—that is, they used a treatment policy strategy. Four (4%) of 99 trials with treatment non-adherence owing to adverse events estimated the treatment effect in a hypothetical setting (ie, the effect as if participants continued treatment despite adverse events), and 19 (79%) of 24 trials where some patients died estimated the treatment effect in a hypothetical setting (ie, the effect as if participants did not die). Conclusions The precise research question being investigated in most trials is unclear, mainly because of a lack of clarity on the approach to handling intercurrent events. Clear reporting of estimands is necessary in trial reports so that all stakeholders, including clinicians, patients and policy makers, can make fully informed decisions about medical interventions. Systematic review registration PROSPERO CRD42021238053.

Section: Strengths and Limitationsmentioning

confidence: 99%

Section: Research In Contextmentioning

confidence: 99%

Evaluating how clear the questions being investigated in randomised trials are: systematic review of estimands

Cro

Rehal

et al. 2022

BMJ

Self Cite

“…As the ICH E9 (R1) guidance has been discussed in detail elsewhere, [4][5][6][7][8][9][10][11][12] our focus is on explaining-through contrasting examples-how researchers can effectively use estimands in their trial planning, design and analysis process to ensure clinical questions are precisely specified and the chance of a successful trial is maximised. In so doing, we hope to convert readers from estimand sceptics to enthusiasts.…”

Section: The Estimand Frameworkmentioning

confidence: 99%

Estimands: bringing clarity and focus to research questions in clinical trials

Clark

Phillips³

et al. 2022

BMJ Open

Precise specification of the research question and associated treatment effect of interest is essential in clinical research, yet recent work shows that they are often incompletely specified. The ICH E9 (R1) Addendum on Estimands and Sensitivity Analysis in Clinical Trials introduces a framework that supports researchers in precisely and transparently specifying the treatment effect they aim to estimate in their clinical trial. In this paper, we present practical examples to demonstrate to all researchers involved in clinical trials how estimands can help them to specify the research question, lead to a better understanding of the treatment effect to be estimated and hence increase the probability of success of the trial.

“…The recent ICH-E9(R1) addendum on estimands 7 provides a strategy to systematically think through the trial objectives and ensure study methods are aligned to the objectives. To achieve a precise definition of the treatment effect the trial aims to estimate, the estimand defines five components [7][8][9][10][11][12][13][14][15] : (i) population; (ii) treatment condition(s) of interest; (iii) outcome measure; (iv) population-level summary (ie, how outcomes under different treatment conditions are to be compared); and (v) how intercurrent events, such as treatment discontinuation or switching, are to be handled.…”

Section: Introductionmentioning

confidence: 99%

Estimands for factorial trials

Morris

Goulão

et al. 2022

Statistics in Medicine

Self Cite

Factorial trials offer an efficient method to evaluate multiple interventions in a single trial, however the use of additional treatments can obscure research objectives, leading to inappropriate analytical methods and interpretation of results. We define a set of estimands for factorial trials, and describe a framework for applying these estimands, with the aim of clarifying trial objectives and ensuring appropriate primary and sensitivity analyses are chosen. This framework is intended for use in factorial trials where the intent is to conduct “two‐trials‐in‐one” (ie, to separately evaluate the effects of treatments A and B), and is comprised of four steps: (i) specifying how additional treatment(s) (eg, treatment B) will be handled in the estimand, and how intercurrent events affecting the additional treatment(s) will be handled; (ii) designating the appropriate factorial estimator as the primary analysis strategy; (iii) evaluating the interaction to assess the plausibility of the assumptions underpinning the factorial estimator; and (iv) performing a sensitivity analysis using an appropriate multiarm estimator to evaluate to what extent departures from the underlying assumption of no interaction may affect results. We show that adjustment for other factors is necessary for noncollapsible effect measures (such as odds ratio), and through a trial re‐analysis we find that failure to consider the estimand could lead to inappropriate interpretation of results. We conclude that careful use of the estimands framework clarifies research objectives and reduces the risk of misinterpretation of trial results, and should become a standard part of both the protocol and reporting of factorial trials.