Statistical power and validity of Ebola vaccine trials in Sierra Leone: a simulation study of trial design and analysis

Abstract

Background Safe and effective vaccines may help end the ongoing Ebola virus disease (EVD) epidemic in West Africa, and mitigate future outbreaks. We evaluate the statistical validity and power of randomized controlled (RCT) and stepped-wedge cluster trial (SWCT) designs in Sierra Leone, where EVD incidence is spatiotemporally heterogeneous, and rapidly declining. Methods We forecasted district-level EVD incidence over the next six months using a stochastic model fit to data from Sierra Leone. We then simulated RCT and SWCT designs in trial populations comprising geographically distinct clusters of high risk, taking into account realistic logistical constraints, as well as both individual-level and cluster-level variation in risk. We assessed false positive rates and power for parametric and nonparametric analyses of simulated trial data, across a range of vaccine efficacies and trial start dates. Findings For an SWCT, regional variation in EVD incidence trends produced inflated false positive rates (up to 0.11 at α=0.05) under standard statistical models, but not when analyzed by a permutation test, whereas all analyses of RCTs remained valid. Assuming a six-month trial starting February 18, 2015, we estimate the power to detect a 90% efficacious vaccine to be between 48% and 89% for an RCT, and between 6.4% and 26% for an SWCT, depending on incidence within the trial population. We estimate that a one-month delay in implementation will reduce the power of the RCT and SWCT by 20% and 49%, respectively. Interpretation Spatiotemporal variation in infection risk undermines the SWCT's statistical power. This variation also undercuts the SWCT's expected ethical advantages over the RCT, because the latter but not the former can prioritize high-risk clusters. Funding US National Institutes of Health, US National Science Foundation, Canadian Institutes of Health Research

Immunology and evolvement of the adenovirus prime, MVA boost Ebola virus vaccine

“…While the declining numbers of cases may not permit direct measures of vaccine efficacy in human subjects [31,32], the current trials along with numerous ongoing and international Phase I and II trials using the ChAd3 Ebola vaccine (with or without MVA boosting) will support ultimate licensure of an Ebola vaccine for human use.…”

Implementing an Ebola Vaccine Study — Sierra Leone

“…Second, once all staff in facilities were enrolled, very limited opportunity existed to expand the sample size, yet declining background rates of Ebola suggested this might be needed. Third, Ebola increasingly occurred in clusters as overall incidence declined; therefore, an imbalance of Ebola risk could easily occur between facilities with vaccinated staff and facilities with unvaccinated staff and lead to lower statistical power and unreliable results (5). Because of these limitations, STRIVE collaborators chose an individually randomized trial of health care and frontline workers assigned to different vaccination times.…”

Statistics and logistics: Design of Ebola vaccine trials in West Africa

“…A special case of the delayed vaccination comparison is the steppedwedge design. 5,6 Under this design, communities or clusters are randomized to a time to receive the vaccine with surveillance of infections on all clusters throughout the trial. The last two options, a historical control or non-randomized concurrent control, were problematic because infection and mortality rates vary over time and by location.…”

“…The stepped-wedge design in the context of this Ebola epidemic was investigated and advocated for in both Ebola vaccine and treatment trials. 5,9 The trial was later redesigned as a comparison between immediate vaccination with rVSV-ZEBOV and vaccination after a 6-month delay.…”