Bayesian model selection techniques as decision support for shaping a statistical analysis plan of a clinical trial: An example from a vertigo phase III study with longitudinal count data as primary endpoint

Abstract

BackgroundA statistical analysis plan (SAP) is a critical link between how a clinical trial is conducted and the clinical study report. To secure objective study results, regulatory bodies expect that the SAP will meet requirements in pre-specifying inferential analyses and other important statistical techniques. To write a good SAP for model-based sensitivity and ancillary analyses involves non-trivial decisions on and justification of many aspects of the chosen setting. In particular, trials with longitudinal count data as primary endpoints pose challenges for model choice and model validation. In the random effects setting, frequentist strategies for model assessment and model diagnosis are complex and not easily implemented and have several limitations. Therefore, it is of interest to explore Bayesian alternatives which provide the needed decision support to finalize a SAP.MethodsWe focus on generalized linear mixed models (GLMMs) for the analysis of longitudinal count data. A series of distributions with over- and under-dispersion is considered. Additionally, the structure of the variance components is modified. We perform a simulation study to investigate the discriminatory power of Bayesian tools for model criticism in different scenarios derived from the model setting. We apply the findings to the data from an open clinical trial on vertigo attacks. These data are seen as pilot data for an ongoing phase III trial. To fit GLMMs we use a novel Bayesian computational approach based on integrated nested Laplace approximations (INLAs). The INLA methodology enables the direct computation of leave-one-out predictive distributions. These distributions are crucial for Bayesian model assessment. We evaluate competing GLMMs for longitudinal count data according to the deviance information criterion (DIC) or probability integral transform (PIT), and by using proper scoring rules (e.g. the logarithmic score).ResultsThe instruments under study provide excellent tools for preparing decisions within the SAP in a transparent way when structuring the primary analysis, sensitivity or ancillary analyses, and specific analyses for secondary endpoints. The mean logarithmic score and DIC discriminate well between different model scenarios. It becomes obvious that the naive choice of a conventional random effects Poisson model is often inappropriate for real-life count data. The findings are used to specify an appropriate mixed model employed in the sensitivity analyses of an ongoing phase III trial.ConclusionsThe proposed Bayesian methods are not only appealing for inference but notably provide a sophisticated insight into different aspects of model performance, such as forecast verification or calibration checks, and can be applied within the model selection process. The mean of the logarithmic score is a robust tool for model ranking and is not sensitive to sample size. Therefore, these Bayesian model selection techniques offer helpful decision support for shaping sensitivity and ancillary analyses in a statistical analysis p...

Efficacy and safety of betahistine treatment in patients with Meniere’s disease: primary results of a long term, multicentre, double blind, randomised, placebo controlled, dose defining trial (BEMED trial)

“…The NB GLMM has fixed effects for treatment group, time (numerical variable for months one to nine), and treatment by time interaction. The model contains normally distributed random intercepts and random slopes associated with time, as well as an offset term for the log-transformed number of evaluated days within each 30 day interval 49. The offset term quantifies the observed information time per 30 days and delivers the denominator for the attack rate.…”

“…The principal model was established by use of data from a previous open non-interventional study36 together with statistical methodology that has been published elsewhere 49…”

Validation of a clinical scoring system for bovine respiratory disease complex diagnosis in preweaned dairy calves using a Bayesian framework

“…The deviance information criteria (DIC) and effective number of models' parameters (pD) were noted as indicators of model fit accounting for overfitting risk (Spiegelhalter et al, 2002). A DIC difference of 5 or more was considered as indicative of a better fit although this is a debatable issue (Adrion and Mansmann, 2012).…”

Effects and tolerability of betahistine in patients with vestibular vertigo: results from the Romanian contingent of the OSVaLD study

“…OSVaLD was an open-label observational study; there are acknowledged limitations to studies of this type, 32 but it was a practical and appropriate format for a multinational trial conducted within the framework of routine care and compliant with the provisions and principles of the Strengthening the Reporting of Observational Studies in Epidemiology methodology. 33 Adrion and Mansmann 34 have recently discussed the merits of Bayesian methodologies for the analysis of clinical trials with longitudinal count data as the primary endpoint, specifically vertigo. This is an approach that may find application in future studies.…”