A commentary on “how to interpret expert judgment assessments of twenty-first century sea-level rise” by Hylke de Vries and Roderik SW van de Wal

Abstract

We clarify key aspects of the evaluation, by de Vries and van de Wal (2015), of our expert elicitation paper on the contributions of ice sheet melting to sea level rise due to future global temperature rise scenarios (Bamber and Aspinall 2013), and extend the conversation with further analysis of their proposed approach for combining expert uncertainty judgments. Aspinall (2013: [BA13]), and welcome this opportunity to clarify the work presented in BA13 and extend the analysis of VW15. The problem of finding a science-based quantification of uncertainty for poorly constrained physical models with large societal impacts deserves high priority in the climate community. This entails crossing discipline boundaries and will take that community outside its usual scientific comfort zone. We therefore salute the authors of VW13 for venturing into this alien terrain and welcome the opportunity to address some of the issues they raise. We thank de Vries and van de Wal (2015: [VW15]) for their detailed consideration of Bamber andThe present commentary discusses certain important and unique attributes of BA13's expert weighting scheme that are misinterpreted in VW15, then addresses the Bconsensus distributionô f VW15, their Blevel of consensus^, and the issue of lognormal fitting elicited data.1 Weighting scheme VW15 state: BThe answers of the experts [in BA13] were weighted using a specialised weighting technique, which involved the self-estimated level of expertise and confidence of the respondents^.

Weighting and aggregating expert ecological judgments

“…A statistical accuracy below 0.05 is often used as a cut‐off point at which an expert is considered statistically inaccurate (i.e., Bamber et al. , Colson and Cooke ). The 0.05 level is often used in meta‐analyses comparing the weighting and aggregation schemes in the Classical Model literature, but can also be used by the analyst as a cut‐off point at which zero weight may be assigned to the expert's judgment.…”

“…However, Bamber et al. () and Colson and Cooke () found that quantile aggregation is much more overconfident than linear pooling (when assessed using the Classical Model's statistical accuracy measure). To investigate these findings, we extend our analysis to compare how the two methods of equal weighted aggregation can affect judgments.…”

Improving expert forecasts in reliability: Application and evidence for structured elicitation protocols

“…Higher values indicate an expert's distribution more closely matches the theoretical distribution. A statistical accuracy below 0.05 is often used as a cut‐off point at which an expert is considered statistically inaccurate (ie, Bamber et al and Colson and Cooke). Information (often referred to as informativeness) under the Classical Model measures the degree to which the distribution supplied is concentrated and to which it deviates from a uniform or log‐uniform distribution (which are considered the least informative distributions). It uses the KL divergence measure, which is scale invariant .…”

Probabilistic Sea Level Projections at the Coast by 2100

“…Both cases yielded a heavy-tailed distribution for the Antarctic ice sheet and hence total sea level. Additional interpretation of Bamber and Aspinall (2013) was made by De Vries and Van de Wal (2015; see also response by Bamber et al 2016) and was consequently used by de Winter et al (2017). Several other probabilistic projections have been made based on a recent Antarctic ice sheet modelling studying by DeConto and Pollard (2016) (e.g., Kopp et al 2017).…”