The analysis of probabilistic fault trees often involves the investigation of events that contribute both to the frequency of the top event and to the uncertainty in this frequency. This paper provides a discussion of three measures of the contribution of an event to the total uncertainty in the top event. These measures are known as uncertainty importance measures. Two of these measures are new developments. Each of the measures is shown to have unique advantages and disadvantages. The three measures are based on, respectively, the expected reduction in the variance of the topevent frequency should the uncertainty in an event be resolved, the same measure based on the log frequency, and a measure based on shifts in the quantiles of the distribution of top-event frequency.

Expert judgment elicitation is often required in probabilistic decision making and the evaluation of risk. One measure of the quality of probability distributions given by experts is calibration--the faithfulness of the probabilities in an empirically verifiable sense. A method of measuring calibration for continuous probability distributions is presented here. A discussion of the impact of using linear rules for combining such judgments is given and an empirical demonstration is given using data collected from experts participating in a large-scale risk study. It is shown by theoretical argument that combining well-calibrated distributions of individual experts using linear rules can only result in reducing calibration. In contrast, it is demonstrated, both by example and empirically, that an equally weighted linear combination of experts who tend to be "overconfident" can produce distributions that are better calibrated than the experts' individual distributions. Using data from training exercises, it is shown that the improvement in calibration is rapid as the number of experts is increased from one to five or six, but there is only modest improvement from increasing the number of experts beyond that point.probability elicitation, linear opinion pools, expert judgment, calibration

No abstract

No abstract

When multiple redundant probabilistic judgments are obtained from subject matter experts, it is common practice to aggregate their differing views into a single probability or distribution. Although many methods have been proposed for mathematical aggregation, no single procedure has gained universal acceptance. The most widely used procedure is simple arithmetic averaging, which has both desirable and undesirable properties. Here we propose an alternative for aggregating distribution functions that is based on the median cumulative probabilities at fixed values of the variable. It is shown that aggregating cumulative probabilities by medians is equivalent, under certain conditions, to aggregating quantiles. Moreover, the median aggregate has better calibration than mean aggregation of probabilities when the experts are independent and well calibrated and produces sharper aggregate distributions for well-calibrated and independent experts when they report a common location-scale distribution. We also compare median aggregation to mean aggregation of quantiles.

A procedure for extending the size of a Latin hypercube sample (LHS) with rank correlated variables is described and illustrated. The extension procedure starts with an LHS of size m and associated rank correlation matrix C and constructs a new LHS of size 2m that contains the elements of the original LHS and has a rank correlation matrix that is close to the original rank correlation matrix C. The procedure is intended for use in conjunction with uncertainty and sensitivity analysis of computationally demanding models in which it is important to make efficient use of a necessarily limited number of model evaluations.

No abstract

No abstract