Uncertainty quantification metrics for whole product life cycle cost estimates in aerospace innovation

Abstract

a b s t r a c tThe lack of defensible methods for quantifying cost estimate uncertainty over the whole product life cycle of aerospace innovations such as propulsion systems or airframes poses a significant challenge to the creation of accurate and defensible cost estimates. Based on the axiomatic definition of uncertainty as the actual prediction error of the cost estimate, this paper provides a comprehensive overview of metrics used for the uncertainty quantification of cost estimates based on a literature review, an evaluation of publicly funded projects such as part of the CORDIS or Horizon 2020 programs, and an analysis of established approaches used by organizations such NASA, the U.S. Department of Defence, the ESA, and various commercial companies. The metrics are categorized based on their foundational character (foundations), their use in practice (state-of-practice), their availability for practice (state-of-art) and those suggested for future exploration (state-of-future). Insights gained were that a variety of uncertainty quantification metrics exist whose suitability depends on the volatility of available relevant information, as defined by technical and cost readiness level, and the number of whole product life cycle phases the estimate is intended to be valid for. Information volatility and number of whole product life cycle phases can hereby be considered as defining multi-dimensional probability fields admitting various uncertainty quantification metric families with identifiable thresholds for transitioning between them. The key research gaps identified were the lacking guidance grounded in theory for the selection of uncertainty quantification metrics and lacking practical alternatives to metrics based on the Central Limit Theorem. An innovative uncertainty quantification framework consisting of; a set-theory based typology, a data library, a classification system, and a corresponding input-output model are put forward to address this research gap as the basis for future work in this field.

Uncertainty-Based Design Optimization and Technology Evaluation: A Review

“…These revolve around the lack of operational meaning of the metrics in theories other than (subjective) probability theory. 116,117 A literature review on uncertainty quantification metrics for whole product life cycle cost in aerospace innovation is presented by Schwabe et al, 118 who indicate that the probability density function is still the most used metric. Additionally, quantification of uncertainty is still largely subjective (i.e.…”

Tolerancing: Managing uncertainty from conceptual design to final product

“…The design of complex systems is more heavily weighted toward the early stages, and the interaction uncertaintydriven by complexityplays a large role in the risk of the process. A discussion of the evolution of uncertainties in cost over a product lifecycle is described by Schwabe et al [261].…”

A framework for geometric quantification and forecasting of cost uncertainty for aerospace innovations

“…A further emphasis discovered was in the exploration of scientific measurement uncertainty. While spatial geometry is commonly used in the engineering, mathematics, natural sciences, big data and meteorology domains, its application to cost estimation appears to be awaiting further investigation [16].…”

“…These generally apply regression based estimation approaches to products which do not address conditions of small data. Conditions of small data arise when few measurement points, little prior experience, no known history low quality data and deep uncertainty are present [12][13][14][15][16]. Two fundamental forecasting approaches exist in forward and inverse uncertainty propagation.…”