A comparison of methods for representing sparsely sampled random quantities.

Abstract

This report discusses the treatment of uncertainties stemming from relatively few samples of random quantities. The importance of this topic extends beyond experimental data uncertainty to situations involving uncertainty in model calibration, validation, and prediction. With very sparse data samples it is not practical to have a goal of accurately estimating the underlying probability density function (PDF). Rather, a pragmatic goal is that the uncertainty representation should be conservative so as to bound a specified percentile range of the actual PDF, say the range between 0.025 and .975 percentiles, with reasonable reliability. A second, opposing objective is that the representation not be overly conservative; that it minimally over-estimate the desired percentile range of the actual PDF. The presence of the two opposing objectives makes the sparse-data uncertainty representation problem interesting and difficult. In this report, five uncertainty representation techniques are characterized for their performance on twenty-one test problems (over thousands of trials for each problem) according to these two opposing objectives and other performance measures. Two of the methods, statistical Tolerance Intervals and a kernel density approach specifically developed for handling sparse data, exhibit significantly better overall performance than the others.

Propagating Stress-Strain Curve Variability in Multi-Material Problems: Temperature-Dependent Material Tests to Plasticity Models to Structural Failure Predictions

“…Investigations in [26,31,32] have concluded 95/90 TIs to be preferable to many other UQ methods tried or critically assessed for estimating, from very sparse sample data, conservative but not overly conservative bounds on the central 95% of response. The other methods tried or critically evaluated include bootstrapping [33], optimized four-parameter Johnson-family distribution fit to the response samples [34], nonparametric kernel density estimation specifically designed for sparse data [35], nonparametric cubic spline PDF fit to the data based on maximum likelihood [36], and Bayesian sparse-data approaches [37].…”

“…This is not the case here; each input stress-strain function of a given material is sampled repeatedly 1 Confidence levels of 75% or 85% are often adequate to manage risk, especially if conservatism from other sources exists in the analysis or results-such as when several sources of uncertainty are present where each involves sparse data conservatively treated with the TI method. Studies in [32] and [42] indicate that when more than one dominant or influential uncertainty sources are sparsely sampled and represented conservatively with TI confidence levels of say >70%, when the conservatively represented uncertainties are combined in linear propagation or aggregation, the individual conservative biases compound to yield substantially greater than 70% confidence of conservative bias in the combined uncertainty estimate.…”

Comparison of Methods for Calculating B-Basis Crack Growth Life Using Limited Tests

“…1) The normal TI method (even though the underlying distribution is not normal or lognormal) gave better coverage probabilities than the bootstrap methods (i.e., about 93%) for smaller sample sizes (8)(9)(10)(11)(12)(13)(14)(15)(16). So, the normal TI method may be a good choice if one is willing to trade off slight underestimation of coverage probability (i.e., 93% instead of 95%) with significantly better B-basis values or lower relative mean margins in comparison to nonparametric method (e.g., about 24-15% for normal TI vs 47-29% for nonparametric).…”

“…Romero et al [13][14][15] performed similar studies to test the performances of the normal TI method, the Pradlwarter-Schuëller kernel density method, the Johnson method, and the nonparametric method. These methods were used to construct two-sided 90% confidence bounds on the probability density function (PDF), ranging between 0.025 and 0.975 percentiles for normal, triangular, and uniform distributions.…”

UQ and V&V techniques applied to experiments and simulations of heated pipes pressurized to failure

“…The likely error that accompanies sparse sampling has a bias toward underestimating the true full-population variance (at least for distribution types and combinations investigated in [9] - [11]). This is unconservative and therefore undesirable for many engineering purposes.…”