Estimation of expectedness: Predictive accuracy of standard therapy outcomes in randomized phase 3 studies in epithelial ovarian cancer

Abstract

BACKGROUNDThe anticipated clinical outcome of the standard/control arm is an important parameter in the design of randomized phase 3 (RP3) trials to properly calculate sample size, power, and study duration. Changing patterns of care or variation in the study population enrolled may lead to a deviation from the initially anticipated outcome. The authors hypothesized that recent changes in patterns of care in epithelial ovarian cancer (EOC) have led to challenges in correctly estimating the outcome of control groups.METHODSA systematic review of the literature was conducted for RP3 trials of EOC published between January 2000 and December 2010. The expected outcome of the control arm as well as the actual outcome achieved by this cohort was collected and a ratio (actual‐over‐expected ratio) was calculated. The estimation of outcome was deemed accurate if the outcome of the control arm was between 0.75 to 1.25 times the anticipated outcome.RESULTSA total of 35 trials were eligible for analysis. Fifteen trials had survival as the primary endpoint whereas 20 had a progression‐based primary endpoint. In total, 12 of 15 trials with a survival‐based endpoint significantly underestimated the outcome of the control arm, whereas only 4 of 20 trials with a progression‐based endpoint did. Studies with a survival endpoint underestimated outcome more frequently than those with a progression endpoint (P<.001).CONCLUSIONSSurvival of the control arm has frequently been underestimated in recent EOC RP3 trials. This underestimation means that the initial statistical assumptions of these trials may have been inaccurate. Underestimating the outcome of the control arm may result in trials being underpowered to demonstrate the absolute benefit they were designed to show. Cancer 2015;121:413–422. © 2014 American Cancer Society.

Targeted survival improvements in clinical trials: Are you an absolutist or relativist?

“…The statistician takes comfort in the fact that by attaining the original number of target events specified in the design, power is maintained to detect the relative difference between the treatment arms whatever the underlying median time to outcome. Expanding on the point made by Castonguay et al 1 in Table 3 of their article, Table 1 presented herein shows how the absolute difference that can be detected with the stated study power varies as the median survival is underestimated by varying amounts; it also demonstrates how the power to detect the original target absolute difference declines. It should be emphasized that the power to detect the relative difference (specified in the hazard ratio) is maintained in all cases.…”

“…The problems with underestimation that Castonguay et al 1 observed in studies with OS as the primary endpoint did not appear to the same extent in studies using PFS or TTP as a primary endpoint. The authors speculate as to the reasons for this in their article and, as they suggest, the introduction of more effective treatments and clinical management into second-line and subsequent treatment must be considered to play a role.…”

“…The design of a clinical trial is a complex business that requires careful consideration of several factors, with one of the most crucial being the assessment of the targeted improvement in the primary outcome measure, which is also one of the principal determinants of sample size. The article by Castonguay et al in the current issue of Cancer 1 reviews the accuracy with which the median progression-free survival (PFS) and overall survival (OS) in the control arm have been estimated in studies of epithelial ovarian cancer over the past 10 years, and in doing so they have highlighted that the design process is even more challenging than was perhaps believed. Their primary observation is that the median OS in the control arm was underestimated by >25% in 80% of the trials they examined (12 of 15 trials); by way of contrast, the median PFS or time to disease progression (TTP) was underestimated by 25% in 20% of the trials studied (4 of 20 trials).…”

“…In those studies in which significant underestimation occurred, the authors noted the detrimental impact on the power to detect absolute differences in these endpoints. 1 …”

Quality of Sample Size Estimation in Trials of Medical Devices: High-Risk Devices for Neurological Conditions as Example

“…This finding is particularly important given the large influence that parameter estimation has on sample size calculation and consequently on the reliability of the study outcomes. For example, the recent study by Castonguay et al reviewed the accuracy with which the median progression-free survival (PFS) and overall survival (OS) in the control arm had been estimated in studies of epithelial ovarian cancer between 2000 and 2010 (21). Their results highlight that PFS and OS of the control arm have frequently been underestimated, which means that the initial statistical assumptions of these trials may have been inaccurate.…”