Prediction Uncertainty Estimation Despite Unidentifiability: An Overview of Recent Developments

Abstract

One of the most important properties of a mathematical model is the ability to make predictions: to predict that which has not yet been measured. Such predictions can sometimes be obtained from a simple simulation, but that requires that the parameters in the model are known from before. In biology, the parameters are usually both not known from before and not identifiable, i.e. the parameter values cannot be determined uniquely from available data. In such cases of unidentifiability, the space of acceptable parameters is large, often infinite in certain directions. For such large spaces, sampling-based approaches that try to characterize the entire space have difficulties. Recently, a new type of alternative approaches that circumvent this characterization problem has been proposed: where one only searches those directions in the space of acceptable parameters that are relevant for the uncertainty of a particular prediction. In this review chapter, these recently proposed methods are compared and contrasted, both regarding theoretical properties, and regarding user experience. The focus is on methods from the field of systems biology, but also methods from biostatistics, pharmacodynamics, and biochemometrics are discussed. The hope is that this review will increase the usefulness and understanding of already proposed methods, and thereby help foster a tradition where predictions only are deemed interesting if their uncertainties have been determined.

Analysis of Practical Identifiability of a Viral Infection Model

“…Nevertheless, above issues are not de facto worrisome barriers if the primary purpose is to evaluate working hypotheses [14, 20]. In this case, finding a working set or a plausible range of parameter values can be deemed sufficient.…”

On the relationship between sloppiness and identifiability

“…The suggestion that sloppiness is a universal -or, more precisely, ubiquitous -property of systems biology models has spurred a debate: should modellers desist from trying to estimate precise values for the parameters and, instead, focus on characterizing model predictions? Exploring this direction, Cedersund and coworkers [13,11,12] coined the term "core predictions" to denote specific model outcomes that can be uniquely determined, even if parameter values cannot. The parameter regions complying with core predictions can be found using optimization, at least for models of moderate size [11].…”

An Introduction to Uncertainty in the Development of Computational Models of Biological Processes

“…Using optimization techniques, they uncovered compensatory mechanisms in a subset of the parameters in the model, leading to the identification of hypothesis to be validated in dedicated experiments. On a more general note, Cedersund [21] provides an overview of various types of predictions that can be made -core predictions allowing to test the quality of the model or poorly determined predictions allowing to improve the overall well-determination of the model parameters. Even predictions that will not be tested experimentally can provide interesting insights into the studied model.…”

“…models of individual organs connected into a multi-organ model). Cedersund [21] subsequently provides an overview of the recent developments in the methods dealing with prediction uncertainty and discusses the price that needs to be paid when bothering with prediction uncertainty.…”

“…The structure of this chapter (and the whole book) follows that of the model development life cycle, starting with model establishment over parameter optimisation and model adaptation and ending with model prediction. [6]; C3, F2 [8]; C4, F1 [9]; C6, F4 [11]; C9, F3 [7]; C10, F5 [14]; C11, F5 [15]; C12, F1 [17]; C13, F16 [16]; C16, F3 [20]; C16, F3 [20]; C17, F3 [21].…”