A multiverse approach to analyzing data from clinical trials

Abstract

Making decisions related to data processing and analysis are crucial steps to be able to extract insights from data in clinical trials. Trial registries (e.g., clinicaltrials.gov) promote transparency about these decisions and ensure many are made in advance. However, data do not always match expectations, potentially requiring deviations from plans. Moreover, even for decisions that can be made in advance and followed, there are often multiple reasonable options for a given decision, options which may impact the conclusions one draws from a study. Methods therefore are needed to address situations in which clinical data do not conform to preregistered plans or involve decisions with multiple reasonable choices to maximize transparency and minimize bias in such situations. This paper proposes a “multiverse” approach to analyzing data from clinical trials when one or more of these situations are true. The multiverse approach involves simultaneously selecting and analyzing all reasonable options for various decision points and presents results across all analysis “universes”. This paper 1) presents an overview of the multiverse approach and shows how it can solve common challenges when analyzing clinical trial data; 2) highlights challenges and decisions commonly encountered when analyzing clinical trials; and 3) demonstrates the multiverse approach using example data from a small clinical trial for posttraumatic stress disorder.