Model distinguishability and inference robustness in mechanisms of cholera transmission and loss of immunity

Abstract

Mathematical models of cholera and waterborne disease vary widely in their structures, in terms of transmission pathways, loss of immunity, and a range of other features. These differences can affect model dynamics, with different models potentially yielding different predictions and parameter estimates from the same data. Given the increasing use of mathematical models to inform public health decision-making, it is important to assess model distinguishability (whether models can be distinguished based on fit to data) and inference robustness (whether inferences from the model are robust to realistic variations in model structure). In this paper, we examined the effects of uncertainty in model structure in the context of epidemic cholera, testing a range of models with differences in transmission and loss of immunity structure, based on known features of cholera epidemiology. We fit these models to simulated epidemic and long-term data, as well as data from the 2006 Angola epidemic. We evaluated model distinguishability based on fit to data, and whether the parameter values, model behavior, and forecasting ability can accurately be inferred from incidence data. In general, all models were able to successfully fit to all data sets, both real and simulated, regardless of whether the model generating the simulated data matched the fitted model. However, in the long-term data, the best model fits were achieved when the loss of immunity structures matched those of the model that simulated the data. Two parameters, one representing person-to-person transmission and the other representing the reporting rate, were accurately estimated across all models, while the remaining parameters showed broad variation across the different models and data sets. The basic reproduction number (ℛ0) was often poorly estimated even using the correct model, due to practical unidentifiability issues in the waterborne transmission pathway which were consistent across all models. Forecasting efforts using noisy data were not successful early in the outbreaks, but once the epidemic peak had been achieved, most models were able to capture the downward incidence trajectory with similar accuracy. Forecasting from noise-free data was generally successful for all outbreak stages using any model. Our results suggest that we are unlikely to be able to infer mechanistic details from epidemic case data alone, underscoring the need for broader data collection, such as immunity/serology status, pathogen dose response curves, and environmental pathogen data. Nonetheless, with sufficient data, conclusions from forecasting and some parameter estimates were robust to variations in the model structure, and comparative modeling can help to determine how realistic variations in model structure may affect the conclusions drawn from models and data.

Comparing alternative cholera vaccination strategies in Maela refugee camp using a transmission model

“…To formally examine parameter uncertainty and practical identifiability, we plotted profile likelihoods for fitted parameters k a , k c , β W , β I , ξ, shown in Supporting Information Figure 9. All parameters were shown to be identifiable (finite confidence bounds), with clear minima in each case, although the uncertainty was comparatively high for β W and ξ, consistent with previous studies showing that these two parameters are often practically unidentifiable [30,42].…”

“…The infectious period, γ, was fixed to 4 days [52,53].Further, because of apparent practical identifiability limitations discovered during the initial parameter estimation (see Supporting Information for more details), we set all human-human transmission parameters denoted β I , equal to each other, and separately we set all human-water transmission parameters, denoted β W , equal to each other. We subsequently set σ, the child-to-adult shedding ratio of pathogen into the environment, to 1 since the β W 's were equal (similar to [42]). Thus the remaining parameters (k a , k c , β I , β W , ξ) were assumed unknown.…”

“…For example, the US Centers for Disease Control and Prevention (CDC) used real-time modeling to predict the effects of vaccination during the 2010 cholera epidemic in Haiti and to anticipate the total numbers of cases and hospitalizations [25]. More broadly, a wide range of cholera transmission models have been developed [26,27,28,29,30,31], accounting for different mechanisms, including spatial dynamics [32,29,33], age structure [34,35], environmental drivers [36,37,38,39], and disease transmission characteristics such as proportion of asymptomatic individuals [38], hyperinfectiousness [40], dose response effects [41,27,29,42], and multiple transmission pathways [28,30]. One particularly relevant modeling analysis examined the impact of one compared with two doses of cholera vaccine in Haiti, Zimbabwe, and Guinea when supplies are limited [43].…”

“…This model is an extension of the classical SIR model (with Susceptible, Infectious, and Recovered compartments to track the total number of individuals at different stages of the disease), with an additional water or environmental compartment representing the concentration of pathogen. The SIWR model has been applied to a range of cholera outbreaks as well as several theoretical studies [32,42,29,28,45]. It has been integrated with a gravity model to determine how distance and population sizes affect the spread of cholera in Haiti [32] and it was used to estimate the basic reproduction number (R 0 ) in a range of settings [29,30,32].…”

A confidence building exercise in data and identifiability: Modeling cancer chemotherapy as a case study

“…In such cases, practically identifiable combinations can often be found (e.g. as seen in models of cholera (Eisenberg et al, 2013; Lee et al, 2016)). Because the degree of practical unidentifiability can range across a spectrum, customarily one defines a threshold for practical identifiability, e.g.…”