Confusing placebo effect with natural history in epilepsy: A big data approach

Abstract

For unknown reasons, placebos reduce seizures in clinical trials in many patients. It is also unclear why some drugs showing statistical superiority to placebo in one trial may fail to do so in another. Using Seizuretracker.com, a patient-centered database of 684,825 seizures, we simulated “placebo” and “drug” trials. These simulations were employed to clarify the sources of placebo effects in epilepsy, and to identify methods of diminishing placebo effects. Simulation 1 included 9 trials with a 6-week baseline and 6-week test period, starting at time 0, 3, 6…24 months. Here, “placebo” reduced seizures regardless of study start time. Regression-to-the-mean persisted only for 3–6 months. Simulation 2 comprised a 6-week baseline and then 2 years of follow-up. Seizure-frequencies continued to improve throughout follow-up. Although the group improved, individuals switched from improvement to worsening and back. Simulation 3 involved a placebo-controlled “drug” trial, to explore methods of placebo-response reduction. An efficacious “drug” failed to demonstrate a significant effect compared with “placebo” (p=0.12), though modifications either in study start-time (p=0.025) or baseline population reduction (p=0.0028) allowed the drug to achieve a statistically significant effect compared with placebo. In epilepsy clinical trials, some seizure reduction traditionally attributed to placebo effect may reflect the natural course of the disease itself. Understanding these dynamics will allow future investigations into optimal clinical trial design and may lead to identification of more effective therapies.

Monte Carlo simulations of randomized clinical trials in epilepsy

“…Using patient reported seizure diaries from SeizureTracker.com,4, 5 we generated a series of virtual patients. The virtual patients were used to form simulated trials, which included patient dropout.…”

“…Each virtual patient was generated using data from one of the world's largest patient reported seizure diary databases, SeizureTracker.com 4, 6. A deidentified, unlinked data export from SeizureTracker was created on June 31, 2016, in accordance with the NIH Office of Human Protection protocol #12301.…”

“…Our group showed recently that the so‐called placebo effect in epilepsy may in fact reflect underlying variability of seizure rates, rather than a true response to placebo 4. If so, then detailed trial parameters are critical factors influencing success or failure.…”

Is seizure frequency variance a predictable quantity?

“…Natural variability in seizure frequency is a relatively unmeasured quantity. However, it may explain a portion of the “placebo effect” in epilepsy trials 5. Gaps in knowledge about this variability hampers interpretation of any randomized clinical trial (RCT) that bases the outcome on seizure frequency changes.…”

“…However, because of the natural variability in seizure frequencies, subjects in the placebo arm may be misidentified as “responders”. Simulations based on 1767 patient seizure diaries showed that many 50%‐responders in RCTs may subsequently become nonresponders (and perhaps subsequently again become responders) due to natural variability 5. This suggests that using the 50%‐responder rate to measure “improvement” is confounded by the noise of natural variability.…”