Implications of Heterogeneity of Treatment Effect for Reporting and Analysis of Randomized Trials in Critical Care

Abstract

Randomized clinical trials (RCTs) are conducted to guide clinicians' selection of therapies for individual patients. Currently, RCTs in critical care often report an overall mean effect and selected individual subgroups. Yet work in other fields suggests that such reporting practices can be improved. Specifically, this Critical Care Perspective reviews recent work on so-called "heterogeneity of treatment effect" (HTE) by baseline risk and extends that work to examine its applicability to trials of acute respiratory failure and severe sepsis. Because patients in RCTs in critical care medicine-and patients in intensive care units-have wide variability in their risk of death, these patients will have wide variability in the absolute benefit that they can derive from a given therapy. If the side effects of the therapy are not perfectly collinear with the treatment benefits, this will result in HTE, where different patients experience quite different expected benefits of a therapy. We use simulations of RCTs to demonstrate that such HTE could result in apparent paradoxes, including: (1) positive trials of therapies that are beneficial overall but consistently harm or have little benefit to low-risk patients who met enrollment criteria, and (2) overall negative trials of therapies that still consistently benefit high-risk patients. We further show that these results persist even in the presence of causes of death unmodified by the treatment under study. These results have implications for reporting and analyzing RCT data, both to better understand how our therapies work and to improve the bedside applicability of RCTs. We suggest a plan for measurement in future RCTs in the critically ill.

Toward Smarter Lumping and Smarter Splitting: Rethinking Strategies for Sepsis and Acute Respiratory Distress Syndrome Clinical Trial Design

“…A recent simulation study suggests that clinically relevant heterogeneity of treatment effect may well exist within critical care trial populations, such that some groups of patients benefit a lot, whereas other groups experience negligible benefits or may even be harmed from treatment (29). Without assessing for heterogeneity of treatment effect, we may mistakenly conclude that a therapy efficacious for some patients is never efficacious because the average treatment effect (the overall result of the clinical trial) is negative.…”

The intensive care medicine research agenda on septic shock

“…This is exemplified by the HES and tight glycaemic control trials that showed harm from the interventions. Second, 'negative' trials may identify important subgroup effects that support more individualized care to a heterogeneous patient population [20]. Third, they may reveal which factors are important in multifaceted interventions.…”

Improving outcomes after critical illness: harder than we thought!

“…Similarly, severe chronic physical co-morbidity can interact with acute physical conditions in ways that may prevent or impair the effectiveness of physical rehabilitation strategies [3,5]. A recent publication on the heterogeneity of treatment effects encourages us to carefully consider how we perform patient selection, stratification and analyses for known risk factors [6]. For example, a study of the use of patient diaries suggested that patients at higher risk of psychological morbidity may be the only ones to benefit from a diary intervention [7].…”