Confronting Deep Uncertainties in Risk Analysis

Cox, Louis Anthony

doi:10.1111/j.1539-6924.2012.01792.x

Cited by 169 publications

(129 citation statements)

References 71 publications

Supporting

Mentioning

123

Contrasting

Unclassified

Order By: Relevance

“…Sense-making, issue and problem formulation, and the process of modelling will also be major foci. The paper continues the arguments begun in French (2013) (for related discussions, see Cox, 2012;Spiegelhalter and Riesch, 2011). In the next section I begin the discussion of sense-making, recognising that it takes place as much in our subconscious thoughts and that formalising this process to build models means that we must cross that vague boundary between intuitive thought and formalised analysis.…”

Section: Introductionmentioning

confidence: 78%

Cynefin: uncertainty, small worlds and scenarios

French

2015

Journal of the Operational Research Society

View full text Add to dashboard Cite

Uncertainty, its modelling and analysis have been discussed across many literatures including statistics and operational research, knowledge management and philosophy: (i) adherents to Bayesian approaches have usually argued that uncertainty should either be modelled by probabilities or resolved by discussion that clarifies meaning; (ii) some have followed Knight in distinguishing between contexts of risk and of uncertainty: the former admitting modelling and analysis through probability; the latter not; (iii) there are also host of approaches in the literatures stemming from Zadeh's concept of a fuzzy set; (iv) theories of sense-making in the philosophy and management literatures see knowledge and uncertainty as opposite extremes of human understanding and discuss the resolution of uncertainty accordingly. Here I provide a personal perspective, taking a Bayesian stance. However, I adopt a softer position than conventional and recognise the concerns in other approaches. In particular, I use the Cynefin framework of decision contexts to reflect on processes of modelling and analysis in statistical, risk and decision analysis. The approach builds on several recent strands of discussion that argue for a convergence of qualitative scenario planning ideas and more quantitative approaches to analysis. I discuss how these suggestions and discussions relate to some earlier thinking on the methodology of modelling and, in particular, the concept of a 'small world' articulated by Savage.

show abstract

Section: Introductionmentioning

confidence: 78%

Cynefin: uncertainty, small worlds and scenarios

French

2015

Journal of the Operational Research Society

View full text Add to dashboard Cite

show abstract

“…Various objective methods have been developed for this purpose: resampling, adaptive boosting, Bayesian information criteria, etc. (46,47). However, the highintensity, low-probability region of the hazard space F ⊗ x most relevant to many risk decisions is dominated by large, infrequent earthquakes.…”

Section: Probabilistic Seismic Hazard Analysismentioning

confidence: 99%

Testing for ontological errors in probabilistic forecasting models of natural systems

Marzocchi

Jordan

2014

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Probabilistic forecasting models describe the aleatory variability of natural systems as well as our epistemic uncertainty about how the systems work. Testing a model against observations exposes ontological errors in the representation of a system and its uncertainties. We clarify several conceptual issues regarding the testing of probabilistic forecasting models for ontological errors: the ambiguity of the aleatory/epistemic dichotomy, the quantification of uncertainties as degrees of belief, the interplay between Bayesian and frequentist methods, and the scientific pathway for capturing predictability. We show that testability of the ontological null hypothesis derives from an experimental concept, external to the model, that identifies collections of data, observed and not yet observed, that are judged to be exchangeable when conditioned on a set of explanatory variables. These conditional exchangeability judgments specify observations with well-defined frequencies. Any model predicting these behaviors can thus be tested for ontological error by frequentist methods; e.g., using P values. In the forecasting problem, prior predictive model checking, rather than posterior predictive checking, is desirable because it provides more severe tests. We illustrate experimental concepts using examples from probabilistic seismic hazard analysis. Severe testing of a model under an appropriate set of experimental concepts is the key to model validation, in which we seek to know whether a model replicates the data-generating process well enough to be sufficiently reliable for some useful purpose, such as long-term seismic forecasting. Pessimistic views of system predictability fail to recognize the power of this methodology in separating predictable behaviors from those that are not.system science | Bayesian statistics | significance testing | subjective probability | expert opinion S cience is rooted in the concept that a model can be tested against observations and rejected when necessary (1). However, the problem of model testing becomes formidable when we consider natural systems. Owing to their scale, complexity, and openness to interactions within a larger environment, most natural systems cannot be replicated in the laboratory, and direct observations of their inner workings are always inadequate. These difficulties raise serious questions about the meaning and feasibility of "model verification" and "model validation" (2), and have led to the pessimistic view that "the outcome of natural processes in general cannot be accurately predicted by mathematical models" (3).Uncertainties in the formal representation of natural systems imply that the forecasting of emergent phenomena such as natural hazards must be based on probabilistic rather than deterministic modeling. The ontological framework for most probabilistic forecasting models comprises two types of uncertainty: an aleatory variability that describes the randomness of the system, and an epistemic uncertainty that characterizes our lack of knowledge about the sy...

show abstract

“…However, recommendations about how to tackle structural uncertainty remain largely elusive. HTA agencies and decision makers may thus be exposed to oversimplifying assessments and recommendations by putting aside complex forms of uncertainty such as structural 'deep' uncertainty [8].…”

Section: Introductionmentioning

confidence: 99%

Towards a New Framework for Addressing Structural Uncertainty in Health Technology Assessment Guidelines

Ghabri¹,

Cleemput

Josselin

2017

PharmacoEconomics

View full text Add to dashboard Cite

International audienceProviding scientific advice and recommendations for public decision making entails identifying, selecting and weighing evidence derived from multiple sources of information through a systematic approach, while taking into account ethical, cultural and societal factors. Integrated in the evaluation process are exchanges between regulatory agencies, private firms, scientific experts and government representatives. In the case of drugs and medical devices, health technology assessment (HTA) agencies are increasingly commissioned to evaluate innovations in order to provide government with recommendations and advice on reimbursement and/or pricing. To undertake this task, HTA agencies [1–6] in Europe and elsewhere have developed methodological guidelines on the economic evaluation of health technologies [7]. One component of these guidelines deals with ways for both manufacturers (pharmaceutical and medical device firms) and HTA agencies evaluators (modelers, economists and public health experts) to address uncertainty. Several types of uncertainty have indeed been identified in HTA: methodological, parameter and structural uncertainty. Most guidelines describe quite well how to deal with the first two categories, although there is still room for improvement. However, recommendations about how to tackle structural uncertainty remain largely elusive. HTA agencies and decision makers may thus be exposed to oversimplifying assessments and recommendations by putting aside complex forms of uncertainty such as struc-tural ‘deep’ uncertainty [8]. The editorial is not intended to promote new approaches to exploring structural uncertainty, rather to emphasizeconcerns related to the topic, such as definition and analysis. Our aim is therefore to highlight the need to renew the analytical framework guidance for HTA

show abstract

Confronting Deep Uncertainties in Risk Analysis

Cited by 169 publications

References 71 publications

Cynefin: uncertainty, small worlds and scenarios

Cynefin: uncertainty, small worlds and scenarios

Testing for ontological errors in probabilistic forecasting models of natural systems

Towards a New Framework for Addressing Structural Uncertainty in Health Technology Assessment Guidelines

Contact Info

Product

Resources

About