Uncertainty and Expert Assessment for Supporting Evaluation of Levees Safety

Abstract

Abstract. In France, levees remain most of the time badly maintained; these long linear structures show signs of weaknesses on numerous occasions. Only incomplete information is usually available. The general lack of data describing the behavior of the infrastructure during unwanted events led to estimate their safety mainly from expert judgment. Thus the ability of the expert to predict the level of functioning of an infrastructure for a type of hazard and its intensity is crucial. An error of judgment can have very serious consequences and the production of reliable information requires the ability of the expert to report accurately the uncertainties in its estimations, as well as associated confidence. In order to meet this need, our research within Incertu project (French Ministry of Ecology funding) aims to produce relevant scientific approaches and tools for the collection and processing reliable experts ¶ statements or combined with a confidence level in the context of uncertain information and input data.

Aggregating predictions from experts: A review of statistical methods, experiments, and applications

“…Political articles focused on predicting presidential outcomes, a categorical target (Graefe, 2015, 2018; Graefe, Armstrong, Jones Jr, & Cuzán, 2014a, 2014b; Hurley & Lior, 2002; Morgan, 2014). Risk‐related targets were continuous and categorical: the probability of structural damage, nuclear fallout, occupational hazards, and balancing power load (Adams et al, 2009; Baecke et al, 2017; Brito et al, 2012; Brito & Griffiths, 2016; Cabello et al, 2012; Craig et al, 2001; Hathout et al, 2016; Jana et al, 2019; Kläs et al, 2010; Mu & Xianming, 1999; Neves & Frangopol, 2008; Ren‐jun & Xian‐zhong, 2002; Wang et al, 2008; Zio, 1996; Zio & Apostolakis, 1997). Public health papers predicted continuous targets over time, like forecasting carcinogenic risk (Evans et al, 1994) and US mortality rates (Alho, 1992).…”

“…Frequentist models for combining forecasts (Adams et al, 2009; Alho, 1992; Alvarado‐Valencia et al, 2017; Baecke et al, 2017; Baldwin, 2015; Borsuk, 2004; Cabello et al, 2012; Cooke et al, 2014; Evans et al, 1994; Franses, 2011; Graefe, Armstrong, Jones Jr, & Cuzán, 2014a; Gu et al, 2016; Hanea et al, 2018; Hathout et al, 2016; Hora & Kardeş, 2015; Hurley & Lior, 2002; Jana et al, 2019; Kläs et al, 2010; Kurowicka et al, 2010; Mak et al, 1996; Morales‐Nápoles et al, 2017; Mu & Xianming, 1999; Ren‐jun & Xian‐zhong, 2002; Seifert & Hadida, 2013; Shin et al, 2013; Wang et al, 2008) were typically convex combinations of expert judgment or linear regression models that included expert judgment as a covariate. Including expert judgment as a covariate in a linear regression model is related to judgmental bootstrapping (Armstrong, 2001b) and the Brunswik lens model (Hammond & Stewart, 2001).…”

“…These fields focus on decision makers and their ability to make predictions from data that cannot easily be collected and fed to a statistical model. Other areas of active research in expert opinion are quality assurance (Kläs et al, 2010), politics (Cai, Lin, Han, Liu, & Zhang, 2016; Graefe, 2015, 2018; Graefe, Armstrong, Jones Jr, & Cuzán, 2014a, 2014b; Hanea, McBride, Burgman, & Wintle, 2018; Satopää, Jensen, Mellers, Tetlock, & Ungar, 2014; Wang & Zhang, 2018), economics (Huang et al, 2016; Mak et al, 1996; Shin et al, 2013), engineering (Brito et al, 2012; Brito & Griffiths, 2016; Craig et al, 2001; Hathout, Vuillet, Peyras, Carvajal, & Diab, 2016; Jin, Lu, & Gan, 2007; Neves & Frangopol, 2008; Ren‐jun & Xian‐zhong, 2002; Tartakovsky, 2007; Wang, Du, & Cao, 2008; Zio, 1996), sports (Gu, Saaty, & Whitaker, 2016), sociology (Adams, White, & Ceylan, 2009; Cabello et al, 2012), meteorological (Abramson, Brown, Edwards, Murphy, & Winkler, 1996), ecological (Borsuk, 2004; Cooke et al, 2014; Failing et al, 2004; Johnson et al, 2018), environmental science (Li, Liu, & Yang, 2012; Mantyka‐Pringle, Martin, Moffatt, Linke, & Rhodes, 2014; Morales‐Nápoles et al, 2017; Zio & Apostolakis, 1997), and public health (Alho, 1992; Evans et al, 1994; Jana, Pramanik, Sahoo, & Mukherjee, 2019; Kurowicka, Bucura, Cooke, & Havelaar, 2010). The diversity and breadth of applications underscore the importance of expert opinion in a wide variety of disciplines.…”

“…Cooke's model also assigned experts a weight of 0 for poor predictive performance, and if an expert's performance was under some user‐set threshold they were excluded from the opinion pool. Cooke's model garnered much attention and has influenced numerous applications of combining expert opinion for forecasting (Bolger & Houlding, 2017; Brito et al, 2012; Clemen, 2008; Clemen & Winkler, 2007; Cooke, 2014, 2015; Cooke et al, 2014; Hanea et al, 2018; Hathout et al, 2016; Hora & Kardeş, 2015; Morales‐Nápoles et al, 2017; Sarin, 2013; Zio, 1996).…”

Return Level Analysis of the Hanumante River Using Structured Expert Judgment: A Reconstruction of Historical Water Levels

“…This model is one of the most applied methods of SEJ, with numerous documented studies [10][11][12]. For example, CM has been previously employed as support in the evaluation of levee safety [13] or for the uncertainty analysis of dike ring failure frequency [14]. Other expert judgment methods are available and have been applied in water related topics.…”