When Should Clinicians Act on Non–Statistically Significant Results From Clinical Trials?

Young, Paul; Nickson, Christopher P; Perner, Anders

doi:10.1001/jama.2020.3508

Cited by 39 publications

(42 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our data can be used to inform the design of future trials and indicate that a sample size of 586 would provide 90% power to detect an absolute difference in the proportion of patients with an unfavourable neurological outcome at day 180 of 13 percentage points, based on a control event rate of 68.1%, and using a two-tailed test. Although some clinicians may choose to implement [ 18 ] conservative oxygen therapy in this patient population based on the consistency of animal data [ 3 ], some observational data [ 4 – 8 ], and trends towards benefits from existing randomized controlled trials [ 6 , 12 ], further larger trials are needed to provide data that are sufficiently robust to generate clear clinical practice recommendations [ 19 ].…”

Section: Resultsmentioning

confidence: 99%

Conservative oxygen therapy for mechanically ventilated adults with suspected hypoxic ischaemic encephalopathy

et al. 2020

View full text Add to dashboard Cite

Purpose Liberal use of oxygen may contribute to secondary brain injury in patients with hypoxic-ischaemic encephalopathy (HIE). However, there are limited data on the effect of different oxygen regimens on survival and neurological disability in HIE patients. Methods We undertook a post-hoc analysis of the 166 patients with suspected HIE enrolled in a trial comparing conservative oxygen therapy with usual oxygen therapy in 1000 mechanically ventilated ICU patients. The primary endpoint for the current analysis was death or unfavourable neurological outcome at day 180. Key secondary outcomes were day 180 mortality, and cause-specific mortality. Results Patients with HIE allocated to conservative oxygen spent less time in the ICU with an SpO 2 ≥ 97% (26 h [interquartile range (IQR) 13–45 vs. 35 h [IQR 19–70], absolute difference, 9 h; 95% CI − 21.4 to 3.4). A total of 43 of 78 patients (55.1%) assigned to conservative oxygen and 49 of 72 patients (68.1%) assigned to usual oxygen died or had an unfavourable neurological outcome at day 180; odds ratio 0.58; 95% CI 0.3–1.12; P = 0.1 adjusted odds ratio 0.54; 95% CI 0.23–1.26; P = 0.15. A total of 37 of 86 patients (43%) assigned to conservative oxygen and 46 of 78 (59%) assigned to usual oxygen had died by day 180; odds ratio 0.53; 95% CI 0.28–0.98; P = 0.04; adjusted odds ratio 0.56; 95% CI 0.25–1.23; P = 0.15. Cause-specific mortality was similar by treatment group. Conclusions Conservative oxygen therapy was not associated with a statistically significant reduction in death or unfavourable neurological outcomes at day 180. The potential for important benefit or harm from conservative oxygen therapy in HIE patients is not excluded by these data. Electronic supplementary material The online version of this article (10.1007/s00134-020-06196-y) contains supplementary material, which is available to authorized users.

show abstract

Section: Resultsmentioning

confidence: 99%

Conservative oxygen therapy for mechanically ventilated adults with suspected hypoxic ischaemic encephalopathy

et al. 2020

View full text Add to dashboard Cite

show abstract

“…A frequentist view of the P value would lead to an interpretation that the new treatment had no effect since the null hypothesis of zero difference cannot be rejected based on a non‐significant P > .05 and exclude the intervention from clinical practice. Many would agree that other factors deserve consideration as well, in particular when the majority of the confidence interval suggests a survival benefit 21 . In a Bayesian analysis, the trial results are treated as a probability distribution.…”

Section: Interpretations Of the Study Resultsmentioning

confidence: 99%

Interpreting the results of clinical trials, embracing uncertainty: A Bayesian approach

Frost

Alexandrou

Schulz

et al. 2020

Acta Anaesthesiol Scand

View full text Add to dashboard Cite

Clinicians must interpret existing knowledge and new evidence as it arises, from well designed, conducted and reported clinical trials to guarantee the best quality of patient care. Clinical evidence is typically collected in an incremental and iterative process where new information is added to existing knowledge. However, the reporting of results from many trials often leads to uncertainty among clinicians on how to interpret a trial's outcomes with the translation of research into practice at times also challenged by prior established practices and beliefs. Traditionally, clinical trials are reported using P values and confidence intervals (CI) relevant to the study hypothesis (most commonly the null hypothesis of zero difference) and effect estimate (often the odds or risk ratio). This approach to inference of trial results is referred to as frequentist and uses data from a single trial in isolation and assigns the probability that the observed outcome has arisen by chance from a hypothetical number of repetitions of the trial (but not that the findings are erroneous or that the hypothesis is false). By convention, a threshold probability of P < .05 given the power of the trial is used as a compromise between type-1 (false positive) and type-2 (false negative) errors to reject the assumption of a chance finding. This approach furthermore leads to a dichotomisation of results that are reported as 'significant' or 'non-significant' purely based on frequentist statistical inference with the 'non-significant' result often receiving a connotation of a 'negative' trial or showing an 'absence' of effect. 1,2 It comes as no surprise that the P value has been criticised with calls made for an alternative approach to inference. [3][4][5][6][7] Using a Bayesian approach of inference, new trial results are considered in the context of existing information (please refer to the Appendix S1 for an explanation of Bayes theorem in an

show abstract

“…Regardless of how the COVID STEROID 2 trial is analysed, some uncertainty is expected to remain even after the results are available. However, as corticosteroids are generally considered safe and are widely used in critically ill patients, the clinical threshold for using a higher dose of corticosteroids in COVID‐19 may be different from thresholds for introducing new, potentially expensive and potentially invasive treatments 42 . Given the current pandemic, it may be considered reasonable to act on probabilities in spite of uncertainty, as has recently been argued for other treatment decisions in the critically ill 42,43 …”

Section: Discussionmentioning

confidence: 99%

Higher vs Lower Doses of Dexamethasone in Patients with COVID‐19 and Severe Hypoxia (COVID STEROID 2) trial: Protocol for a secondary Bayesian analysis

Granholm

Munch

Myatra

et al. 2021

Acta Anaesthesiol Scand

Self Cite

View full text Add to dashboard Cite

Background Coronavirus disease 2019 (COVID‐19) can lead to severe hypoxic respiratory failure and death. Corticosteroids decrease mortality in severely or critically ill patients with COVID‐19. However, the optimal dose remains unresolved. The ongoing randomised COVID STEROID 2 trial investigates the effects of higher vs lower doses of dexamethasone (12 vs 6 mg intravenously daily for up to 10 days) in 1,000 adult patients with COVID‐19 and severe hypoxia. Methods This protocol outlines the rationale and statistical methods for a secondary, pre‐planned Bayesian analysis of the primary outcome (days alive without life support at day 28) and all secondary outcomes registered up to day 90. We will use hurdle‐negative binomial models to estimate the mean number of days alive without life support in each group and present results as mean differences and incidence rate ratios with 95% credibility intervals (CrIs). Additional count outcomes will be analysed similarly and binary outcomes will be analysed using logistic regression models with results presented as probabilities, relative risks and risk differences with 95% CrIs. We will present probabilities of any benefit/harm, clinically important benefit/harm and probabilities of effects smaller than pre‐defined clinically minimally important differences for all outcomes analysed. Analyses will be adjusted for stratification variables and conducted using weakly informative priors supplemented by sensitivity analyses using sceptic priors. Discussion This secondary, pre‐planned Bayesian analysis will supplement the primary, conventional analysis and may help clinicians, researchers and policymakers interpret the results of the COVID STEROID 2 trial while avoiding arbitrarily dichotomised interpretations of the results. Trial registration ClinicalTrials.gov: NCT04509973; EudraCT: 2020‐003363‐25.

show abstract

When Should Clinicians Act on Non–Statistically Significant Results From Clinical Trials?

Cited by 39 publications

References 8 publications

Conservative oxygen therapy for mechanically ventilated adults with suspected hypoxic ischaemic encephalopathy

Conservative oxygen therapy for mechanically ventilated adults with suspected hypoxic ischaemic encephalopathy

Interpreting the results of clinical trials, embracing uncertainty: A Bayesian approach

Higher vs Lower Doses of Dexamethasone in Patients with COVID‐19 and Severe Hypoxia (COVID STEROID 2) trial: Protocol for a secondary Bayesian analysis

Contact Info

Product

Resources

About