Practical limits for reverse engineering of dynamical systems: a statistical analysis of sensitivity and parameter inferability in systems biology models

Abstract

The size and complexity of cellular systems make building predictive models an extremely difficult task. In principle dynamical time-course data can be used to elucidate the structure of the underlying molecular mechanisms, but a central and recurring problem is that many and very different models can be fitted to experimental data, especially when the latter are limited and subject to noise. Even given a model, estimating its parameters remains challenging in real-world systems. Here we present a comprehensive analysis of 180 systems biology models, which allows us to classify the parameters with respect to their contribution to the overall dynamical behaviour of the different systems. Our results reveal candidate elements of control in biochemical pathways that differentially contribute to dynamics. We introduce sensitivity profiles that concisely characterize parameter sensitivity and demonstrate how this can be connected to variability in data. Systematically linking data and model sloppiness allows us to extract features of dynamical systems that determine how well parameters can be estimated from time-course measurements, and associates the extent of data required for parameter inference with the model structure, and also with the global dynamical state of the system. The comprehensive analysis of so many systems biology models reaffirms the inability to estimate precisely most model or kinetic parameters as a generic feature of dynamical systems, and provides safe guidelines for performing better inferences and model predictions in the context of reverse engineering of mathematical models for biological systems.

Topological sensitivity analysis for systems biology

“…As well as the values of individual parameters, we may also be interested in the dependencies between parameters. In particular, the related concepts of sloppiness and identifiability in biological models have recently received much attention-in the context of possible biological significance and for optimal experimental design (12,(36)(37)(38)(39). Fig.…”

“…PSA techniques can usually be classified as (i) local analyses, in which we identify a single "optimal" vector of parameter values, and then quantify the degree to which small perturbations to these values change our conclusions or predictions; or (ii) global analyses, where we consider an ensemble of parameter vectors (e.g., samples from the posterior distribution in the Bayesian formalism) and quantify the corresponding variability in the model's output. Although several approaches fall within these categories (10)(11)(12), all implicitly condition on a particular model architecture. Here we present a method for performing sensitivity analyses for ordinary differential equation (ODE) models where the architecture of these models is not perfectly known, which is likely to be the case for all realistic complex systems.…”

How to deal with parameters for whole-cell modelling

“…For sufficiently simple models (those where the likelihood function in concave around a single maximum), even global uncertainty statements can be made. Experience suggests that this latter case is the exception rather than the rule for dynamical systems in cell and molecular biology [54,58]. Some approaches, such as profile-likelihood methods, try to assess the uncertainty for each parameter [59,60], which may hold particular appeal for those interested in inferring specific parameters with accuracy.…”

A general moment expansion method for stochastic kinetic models

“…28,29 Such analyses allow us to quantify how rapidly the outputs of our model change as we vary its parameters, which can provide insights into the robustness of the model and the relative influence that each parameter has upon the model's behaviour. However, sensitivity analyses of stochastic models can be difficult and/or computationally costly, 30,31 and often involve simulating many times in order to obtain Monte Carlo estimates of sensitivity coefficients.…”