PISC Expert Team White Paper: Toward a Consistent Standard of Evidence When Evaluating the Efficacy of an Experimental Treatment From a Randomized, Active-Controlled Trial

Peterson, Patrick; Carroll, Kevin; Chuang‐Stein, Christy; Ho, Yu-Yun; Jiang, Qi; Li, Gang; Sanchez, Matilde; Sax, Rick; Wang, Yongcheng; Snapinn, Steven

doi:10.1198/sbr.2010.09016

Cited by 13 publications

(19 citation statements)

References 25 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, when NI trials are designed to preserve a percentage of effect of the standard treatment, the rate of approval of “insufficiently effective” treatments not meeting this requirement should also be considered when evaluating bio-creep risk. Peterson et al (2010) argued against preservation of effect, calling standards based on percent preservation “inherently arbitrary and lacking in objective clinical or scientific justification.” By contrast, Fleming et al (2011) maintained that a stricter standard is clinically justified when randomization to placebo is considered unethical due to availability of effective therapies, especially when the active comparator has demonstrated efficacy with respect to irreversible morbidity or mortality. Although the debate about preservation of effect for regulatory approval remains active, we will address the scientifically well motivated setting where investigators wish to control the risk of approval of insufficiently effective therapies at a level of 0.025.…”

Section: Introductionmentioning

confidence: 99%

A Simulation Study Evaluating Bio-Creep Risk in Serial Noninferiority Clinical Trials for Preservation of Effect

Odem‐Davis

Fleming

2015

Statistics in Biopharmaceutical Research

View full text Add to dashboard Cite

In non-inferiority trials, acceptable efficacy of an experimental treatment is established by ruling out some defined level of reduced effect relative to an effective active control standard. Serial use of non-inferiority trials may lead to newly approved therapies that provide meaningfully reduced levels of benefit; this phenomenon is called bio-creep. Simulations were designed to facilitate understanding of bio-creep risk when approval of an experimental treatment with efficacy less than some proportion of the effect of the active control treatment would constitute harm, such as when new antibiotics that are meaningfully less effective than the most effective current antibiotic would be used for treatment of Community-Acquired Bacterial Pneumonia. In this setting, risk of approval of insufficiently effective therapies may be great, even when the standard treatment effect satisfies constancy across trials. Modifiable factors contributing to this manifestation of bio-creep included the active control selection method, the non-inferiority margin, and bias in the active control effect estimate. Therefore, when non-inferiority testing is performed, the best available treatment should be used as the standard, and margins should be based on the estimated effect of this control, accounting for the variability and for likely sources of bias in this estimate, and addressing the importance of preservation of some portion of the standard’s effect.

show abstract

Section: Introductionmentioning

confidence: 99%

A Simulation Study Evaluating Bio-Creep Risk in Serial Noninferiority Clinical Trials for Preservation of Effect

Odem‐Davis

Fleming

2015

Statistics in Biopharmaceutical Research

View full text Add to dashboard Cite

show abstract

“…However, in some discussions only a ‘discounting’ factor is applied with no preservation of effect. [7, 8] Snapinn and Jiang (2008) argued in favor of the latter setting and suggested only a discounting factor incorporating NI trial estimate bias (“assay sensitivity”) as well as historical estimate bias. [8] Fleming et al (2011) emphasized the need for preservation of effect, in addition to addressing the risk for bias, especially when the active comparator has demonstrated efficacy with respect to irreversable morbidity or mortality.…”

Section: Current Margin Methods For Assessment Of Non-inferiority Amentioning

confidence: 99%

Adjusting for Unknown Bias in Noninferiority Clinical Trials

Odem‐Davis

Fleming

2013

Statistics in Biopharmaceutical Research

View full text Add to dashboard Cite

Evaluation of non-inferiority is based on ruling out a threshold for what would constitute unacceptable loss of efficacy of an experimental treatment relative to an active comparator “Standard”. This threshold, the “non-inferiority margin”, is often based on preservation of a percentage of Standard’s effect. To obtain an estimate of this effect to be used in the development of the “non-inferiority margin”, data are needed from earlier trials comparing Standard to placebo if the non-inferiority trial does not have a placebo arm. This approach often provides a biased over-estimate of Standard’s true effect in the setting of the current non-inferiority study. We describe two commonly used non-inferiority margin methods that adjust for this bias, the two-confidence interval (95-95) and the Synthesis margins. However, the added ‘variance inflation’ adjustment made by 95-95 margin diminishes with increasing information from historical trial(s), and the Synthesis margin is based on a strong assumption that the relative bias is known. We introduce an alternative “Bias-adjusted” margin addressing vulnerabilities of each by attenuating the estimate and by accounting for uncertainty in the true level of bias. Examples and asymptotic estimates of non-inferiority hypothesis rejection rates in the proportional hazards setting are used to compare methods.

show abstract

“…Some argue that the preservation of a set fraction or amount of the Standard effect inappropriately creates a higher standard than would be required in a placebo-controlled trial [27,28]. However, this need for preservation of effect, resulting in a higher bar for efficacy for new interventions, is clinically and ethically justified once clinically meaningful benefit has been achieved by Standard in settings of irreversible morbidity or mortality [29].…”

Section: Consideration Bmentioning

confidence: 99%

Some essential considerations in the design and conduct of non-inferiority trials

et al. 2011

View full text Add to dashboard Cite

Background Suppose a standard therapy (Standard) has been established to provide a clinically important reduction in risk of irreversible morbidity or mortality. In that setting, the safety and efficacy of an experimental intervention likely would be assessed in a clinical trial providing a comparison with Standard rather than a placebo arm. Such a trial often is designed to assess whether the efficacy of the experimental intervention is not unacceptably worse than that of Standard, and is called a non-inferiority trial. Formally, the non-inferiority trial usually is designed to rule out a non-inferiority margin, defined as the minimum threshold for what would constitute an unacceptable loss of efficacy. Purpose Even though the literature has many important articles identifying various approaches to the design and conduct of non-inferiority trials, confusion remains especially regarding key considerations for selecting the non-inferiority margin. The purpose of this article is to provide improved clarity regarding these considerations. Methods We present scientific insights into many factors that should be addressed in the design and conduct of non-inferiority trials to enhance their integrity and reliability, and provide motivation for key considerations that guide the selection of non-inferiority margins. We also provide illustrations and insights from recent experiences. Results Two considerations are essential, and should be addressed in separate steps, in the formulation of the non-inferiority margin. First, the margin should be formulated using adjustments to account for bias or lack of reliability in the estimate of the effect of Standard in the non-inferiority trial setting. Second, the non-inferiority margin should be formulated to achieve preservation of an appropriate percentage of the effect of Standard. Limitations The considerations, in particular regarding the importance of preservation of effect, might not apply to settings where it would be ethical as well as clinically relevant to include both Standard and placebo arms in the trial for direct comparisons with the experimental intervention arm. Conclusions Non-inferiority trials with non-rigorous margins allow substantial risk for accepting inadequately effective experimental regimens, leading to the risk of erosion in quality of health care. The design and conduct of non-inferiority trials, including selection of non-inferiority margins, should account for many factors that can induce bias in the estimated effect of Standard in the non-inferiority trial and thus lead to bias in the estimated effect of the experimental treatment, for the need to ensure the experimental treatment preserves a clinically acceptable fraction of Standard's effect, and for the particular vulnerability of the integrity of a non-inferiority trial to the irregularities in trial conduct. Due to the inherent uncertainties in non-inferiority trials, alternative designs should be pursued whenever possible.

show abstract

PISC Expert Team White Paper: Toward a Consistent Standard of Evidence When Evaluating the Efficacy of an Experimental Treatment From a Randomized, Active-Controlled Trial

Cited by 13 publications

References 25 publications

A Simulation Study Evaluating Bio-Creep Risk in Serial Noninferiority Clinical Trials for Preservation of Effect

A Simulation Study Evaluating Bio-Creep Risk in Serial Noninferiority Clinical Trials for Preservation of Effect

Adjusting for Unknown Bias in Noninferiority Clinical Trials

Some essential considerations in the design and conduct of non-inferiority trials

Contact Info

Product

Resources

About