Comparison of Error Rates in Single-Arm Versus Randomized Phase II Cancer Clinical Trials

Abstract

A B S T R A C T PurposeTo improve the understanding of the appropriate design of phase II oncology clinical trials, we compared error rates in single-arm, historically controlled and randomized, concurrently controlled designs. Patients and MethodsWe simulated error rates of both designs separately from individual patient data from a large colorectal cancer phase III trials and statistical models, which take into account random and systematic variation in historical control data. ResultsIn single-arm trials, false-positive error rates (type I error) were 2 to 4 times those projected when modest drift or patient selection effects (eg, 5% absolute shift in control response rate) were included in statistical models. The power of single-arm designs simulated using actual data was highly sensitive to the fraction of patients from treatment centers with high versus low patient volumes, the presence of patient selection effects or temporal drift in response rates, and random small-sample variation in historical controls. Increasing sample size did not correct the over optimism of single-arm studies. Randomized two-arm design conformed to planned error rates. ConclusionVariability in historical control success rates, outcome drifts in patient populations over time, and/or patient selection effects can result in inaccurate false-positive and false-negative error rates in single-arm designs, but leave performance of the randomized two-arm design largely unaffected at the cost of 2 to 4 times the sample size compared with single-arm designs. Given a large enough patient pool, the randomized phase II designs provide a more accurate decision for screening agents before phase III testing.

Correlation between overall survival and other endpoints in clinical trials of second-line chemotherapy for patients with advanced gastric cancer

“…These results suggest that a single-arm phase II study with a primary endpoint of ORR may not be adequate to evaluate the efficacy of second-line chemotherapy for AGC. Randomised phase II studies that compare standard treatments and investigational treatments may be better methods of screening for effective treatments to include within phase III trials [26].…”

RECIST: No Longer the Sharpest Tool in the Oncology Clinical Trials Toolbox—Point

“…The use of categorical response criteria magnifies the impact of measurement variability, whereas comparisons of continuous data minimize this impact. Tang and colleagues showed that, in single-arm phase II trials with RECIST response rate as the primary endpoint, small errors in historical response rate assumptions can result in a 2-to 4-fold increase in the rate of phase II trials that are falsely interpreted as positive (24). Randomization could minimize erroneous conclusions based on RECIST-based endpoints, but the power of such trials would lag behind the power of trials that use comparisons of continuous data.…”

Tumor Burden Modeling Versus Progression-Free Survival for Phase II Decision Making

“…There are many possible designs of a Phase II oncology trial (1), and randomized Phase II trials are often recommended (2)(3)(4)(5). In such randomized trials, patients' tumors are assessed through radiographic imaging, and Response Evaluation Criteria in Solid Tumors (RECIST) criteria (6) applied for determination of tumor burden, tumor response, and progression-free survival (PFS), defined as the earlier of the occurrence of progressive disease or death.…”