Which uncertainty? Using expert elicitation and expected value of information to design an adaptive program

Runge, Michael C.; Converse, Sarah J.; Lyons, James E.

doi:10.1016/j.biocon.2010.12.020

Cited by 376 publications

(460 citation statements)

References 21 publications

Supporting

Mentioning

445

Contrasting

Unclassified

Order By: Relevance

“…Notably, increasing hospitalization and reducing hospital transmission were rarely optimal interventions, even at 100% effectiveness. We calculated the expected value of perfect information (EVPI) (10,11), which quantifies the maximum achievable improvement in management that could be obtained by identifying a single model as "best" before the implementation of specific decisions (Materials and Methods has a formal definition). The EVPI analysis showed that the expected improvement in management outcomes caused by resolving all model-specific uncertainties is an 11% reduction in caseload.…”

Section: Resultsmentioning

confidence: 99%

“…The EVPI analysis showed that the expected improvement in management outcomes caused by resolving all model-specific uncertainties is an 11% reduction in caseload. We further conducted an analysis of the expected value of partial information (EVPXI) (10), which quantifies the expected improvement in management performance by resolving a subset of uncertainties. In particular, we quantified the relative contribution of uncertainty about model structure (SEIR, SEIHR, SEIFR, or SEIHFR) and caseload projection (models that projected low, low intermediate, high intermediate, or high case burden) to expected management outcomes.…”

Section: Resultsmentioning

confidence: 99%

“…We calculated EVPI, which quantifies the maximum achievable improvement in management that could be obtained by resolving uncertainties before the implementation of specific decisions (10,11). It is quantified as EVPI = X n j=1 p j À opt a C a,j Á − opt a X n j=1 p j C a,j , [1] where n is the total number of models, C a,j represents management performance (i.e., caseload in this study) associated with taking intervention a under model j, p j is the weight associated with model j (i.e., the belief that model j is the true model; subject to the constraint that the p j sum to one), and opt a indicates the optimum over all interventions (10,11). In this initial analysis, we weighed the models equally.…”

Section: Methodsmentioning

confidence: 99%

“…We projected the caseload under five management interventions and identified the optimum management for each model. We then used value of information (VoI) (10,11) analyses to quantify the potential improvement in caseload outcomes …”

mentioning

confidence: 99%

See 3 more Smart Citations

Essential information: Uncertainty and optimal control of Ebola outbreaks

Bjørnstad

Ferrari

et al. 2017

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

View full text Add to dashboard Cite

Early resolution of uncertainty during an epidemic outbreak can lead to rapid and efficient decision making, provided that the uncertainty affects prioritization of actions. The wide range in caseload projections for the 2014 Ebola outbreak caused great concern and debate about the utility of models. By coding and running 37 published Ebola models with five candidate interventions, we found that, despite this large variation in caseload projection, the ranking of management options was relatively consistent. Reducing funeral transmission and reducing community transmission were generally ranked as the two best options. Value of information (VoI) analyses show that caseloads could be reduced by 11% by resolving all model-specific uncertainties, with information about model structure accounting for 82% of this reduction and uncertainty about caseload only accounting for 12%. Our study shows that the uncertainty that is of most interest epidemiologically may not be the same as the uncertainty that is most relevant for management. If the goal is to improve management outcomes, then the focus of study should be to identify and resolve those uncertainties that most hinder the choice of an optimal intervention. Our study further shows that simplifying multiple alternative models into a smaller number of relevant groups (here, with shared structure) could streamline the decision-making process and may allow for a better integration of epidemiological modeling and decision making for policy.value of information | VoI | epidemiological outbreak management | decision making T he devastating 2014 Ebola outbreak in West Africa is the largest ever recorded (1, 2). It resulted in 28,646 cases and 11,323 deaths by March 27, 2016 (WHO report; apps.who.int/ebola/ ebola-situation-reports) and engendered an outpouring of concern for those affected. A large number of epidemiological models were developed and published (2-4). To date, we have identified 55 published Ebola models. Most of these models (50 of 55) projected caseloads as the preferred way to predict epidemic trajectory. However, caseload projections varied widely between models, drawing a great deal of attention and causing intense debate (5, 6).Caseload projection is critical for predicting the size of an epidemic and planning management efforts, and it can vary from model to model for several reasons, such as differences in model structure, parameterization, and other assumptions. Despite model-specific variations in caseload projections, a critical question for decision making is whether different models lead to different management recommendations or different rankings of alternative management actions. If all models agree on the optimal management, then differences in projections are not a critical concern for decision making. Otherwise, if models disagree with respect to the ranking of management recommendations, then the optimal intervention is modelspecific, which means that policymakers face the question of which model(s) to rely on to make management decisions;...

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Essential information: Uncertainty and optimal control of Ebola outbreaks

Bjørnstad

Ferrari

et al. 2017

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

View full text Add to dashboard Cite

show abstract

“…For models, an adaptive strategy improves predictions as foundational flaws in modeling assumptions or procedures are discovered and/or as tests of model estimates become available through acquisition of empirical data (e.g., Bromberg et al 2011;Cianfrani et al 2010;Jones 2012). This approach serves to improve prediction by reducing structural and parametric uncertainty arising from incomplete (or inaccurate) knowledge about the modeled system (Runge et al 2011).…”

Section: Range Model Approaches Pitfalls and Model Selectionmentioning

confidence: 99%

Weighing empirical and hypothetical evidence for assessing potential invasive species range limits: a review of the case of Burmese pythons in the USA

Engeman

Avery

Jacobson

2014

Environ Sci Pollut Res

View full text Add to dashboard Cite

Range expansion potential is an important consideration for prioritizing management actions against an invasive species. Understanding the potential for range expansion by invasive reptiles such as the Burmese python can be challenging, because the lack of knowledge on fundamental physiological and behavioral constraints initially forces reliance on modeling to predict hypothetical invasive range potential. Hypothetical predictions for Burmese python range limits in the USA have been highly divergent, from only extreme South Florida and the extreme southern Gulf edge of Texas to a broad swath over the southern third of the continental USA. Empirical observations on python thermal tolerances and behavioral abilities to cope with more temperate temperatures became evident during a cold spell in December 2009-January 2010. We review and highlight important considerations for improving invasive range estimation methodology, deciding between competing range predictions, and the importance of having, and applying, empirical data to aid in decision making.

show abstract

Uncertainty in population estimates: A meta‐analysis for petrels

Bird

Woodworth

Fuller

et al. 2021

Ecol Sol and Evidence

View full text Add to dashboard Cite

1. Population estimates are commonly generated and used in conservation science.All estimates carry inherent uncertainty, but little attention has been given to when and how this uncertainty limits their use. This requires an understanding of the specific purposes for which population estimates are intended, an assessment of the level of uncertainty each purpose can tolerate, and information on current uncertainty.2. We conducted a review and meta-analysis for a widespread group of seabirds, the petrels, to better understand how and why population estimates are being used.Globally petrels are highly threatened, and aspects of their ecology make them difficult to survey, introducing high levels of uncertainty into population estimates.3. We found that by far the most common intended use of population estimates was to inform status and trend assessments, while less common uses were trialling methods to improve estimates and assessing threat impacts and conservation outcomes.4. The mean coefficient of variation for published estimates was 0.17 (SD = 0.14), with no evidence that uncertainty has been reduced through time. As a consequence of this high uncertainty, when we simulated declines equivalent to thresholds commonly used to trigger management, only 5% of studies could detect significant differences between population estimates collected 10 years apart for populations declining at a rate of 30% over three generations. Reporting of uncertainty was variable with no dispersion statistics reported with38% of population estimates and most not reporting key underlying parameters: nest numbers/density and nest occupancy. We also found no correlation between uncertainty in petrel population estimates and either island size, body size or species threat status -potential predictors of uncertainty. 6. Key recommendations for managers are to be mindful of uncertainty in past population estimates if aiming to collect contemporary estimates for comparison, to report uncertainty clearly for new estimates, and to give careful consideration to whether a proposed estimate is likely to achieve the requisite level of certainty for the This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

Which uncertainty? Using expert elicitation and expected value of information to design an adaptive program

Cited by 376 publications

References 21 publications

Essential information: Uncertainty and optimal control of Ebola outbreaks

Essential information: Uncertainty and optimal control of Ebola outbreaks

Weighing empirical and hypothetical evidence for assessing potential invasive species range limits: a review of the case of Burmese pythons in the USA

Uncertainty in population estimates: A meta‐analysis for petrels

Contact Info

Product

Resources

About