The Primary Outcome Fails — What Next?

Abstract

A well-designed trial derives its credibility from the inclusion of a prespecified, a priori hypothesis that helps its authors avoid making potentially false positive claims on the basis of an exploratory analysis of the data. Nevertheless, an unreasonable yet widespread practice is the labeling of all randomized trials as either positive or negative on the basis of whether the P value for the primary outcome is less than 0.05. This view is overly simplistic. P values should be interpreted as a continuum wherein the smaller the P value, the greater the strength of the evidence for a real treatment effect. 1-3 Confidence intervals are also useful in indicating the range of uncertainty around the estimated treatment effect. Moreover, the interpretation of any trial should depend on the totality of the evidence (i.e., the primary, secondary, and safety outcomes), not just a single end point.Herein we outline the thought processes to follow when the primary outcome of a trial is first perceived as negative. In a subsequent issue of the Journal, we will review the questions to ask when the outcomes of a trial appear to be positive. We illustrate our points with examples from trials -primarily from cardiovascular studies, which constitute our sphere of expertise -but the underlying issues apply to the whole of medicine. K ey Ques tions W hen the Pr im a r y Ou t come Fa il sThe failure to achieve the 5% level of significance is certainly not promising for the test treatment. Typical first reactions include the following: What went wrong? Is the treatment truly ineffective? Are there glimmers of hope? What next? Addressing the 12 questions below provides a path forward (Table 1). Is There Some Indication of Potential Benefit?Whether a signal of treatment benefit (a "trend") should be inferred from a P value greater than 0.05 requires thoughtful consideration. When the primary findings of a trial are completely neutral, interpretation is straightforward. For instance, when the PERFORM trial of terutroban versus aspirin in patients with ischemic stroke (see box for a list of the complete names of all trials mentioned in this article) 4 showed no significant between-group difference with respect to the composite primary outcome -ischemic stroke, myocardial infarction, or other vascular cause of death (hazard ratio, 1.02; 95% confidence interval [CI], 0.94 to 1.12)the trial was stopped early because of futility, and no safety advantages were detected for terutroban. These findings support the interpretation of a "negative trial."In contrast, in the TORCH trial, 5 in which the effects of salmeterol plus flutica-

National randomized controlled trial of virtual house calls for Parkinson disease

“…Additional possibilities include the very high proportion of individuals who were already receiving care from a PD specialist at baseline, the high satisfaction with that care at baseline, the required reliance (due to state licensing laws) on local clinicians to implement the specialist's recommendations, and the choice of measure used (the patients' global impression of change was significantly favorable for those receiving virtual visits). Beyond the primary efficacy outcome measure, 26 the secondary outcomes highlight the care model's benefits and are consistent with most of the Institute of Medicine's recommendations for improving care.…”

The intensive care medicine research agenda on septic shock

“…However, these so-called 'negative' trials have nonetheless generated considerable new knowledge leading both to scientific advances and important changes in clinical practice [16,17].…”

Evidence in Medicine: Math Versus Biology!

“…As examples of the former, the biologic observations encountered in rcts (such as disease progression or relapse, patient survival, and quality of life) are often represented by bio-statistical measures that include estimates of effect size and the uncertainties involved (for example, type i and ii errors, p values, and 95% confidence intervals) 6,7 . As well, meta-analyses often use mathematical measures that extrapolate efficacy from individual rcts to compute compound measures of effect size and uncertainty.…”

“…Nonetheless, a working knowledge of the underlying statistical methods is necessary for clinicians to properly interpret the current evidence. Furthermore, knowledge that is more in-depth is also often required to properly interpret the clinical evidence and avoid the pitfalls of mere binary interpretation of study outcomes as simply positive or negative 6,7 .…”