Bridging the Gap between Mechanistic Biological Models and Machine Learning Surrogates

Abstract

Mechanistic models have been used for centuries to describe complex interconnected processes, including biological ones. As the scope of these models has widened, so have their computational demands. This complexity can limit their suitability when running many simulations or when real-time results are required. Surrogate machine learning models can be used to approximate the behaviour of complex mechanistic models, and once built, their computational demands are several orders of magnitude lower. This paper provides an overview of the relevant literature, both from an applicability and a theoretical perspective. For the latter, the paper focuses on the design and training of the underlying machine learning models. Application-wise, we show how machine learning surrogates have been used to approximate different mechanistic models. We present a perspective on how these approaches can be applied to models representing biological processes with potential industrial applications (e.g., metabolism and whole-cell modelling) and show why surrogate machine learning models may hold the key to making the simulation of complex biological models possible using a typical desktop computer.

Modeling With Uncertainty Quantification Identifies Essential Features of a Non-Canonical Algal Carbon-Concentrating Mechanism

“…By training a surrogate model on the parameter space of mechanistic biological models, we can understand and account for high-dimensional uncertainty in model parameters. Metabolic modeling in general has been highlighted as a particularly promising application of surrogate modeling, since metabolic modeling has biotechnological potential but is challenged by the complexity of metabolism and by the “trial and error” process which is often required to produce a working metabolic model (21). Surrogate modeling has found uses in dynamic flux balance analysis and process modeling for bioprocesses (51, 52).…”

“…Our methods may be particularly valuable for models that have poorly-defined parameters or are extremely computationally expensive. For example, the implementation of surrogate modeling described here could alleviate current limitations in interpreting reaction-diffusion models and genome-scale metabolic models (21).…”

A review of mechanistic learning in mathematical oncology

“…A related concept is the generation of vast amounts of "synthetic" training data (65) based on a small set of "original" data points. While synthetic training data can improve the accuracy of many learning-based systems, care needs to be taken to prevent encoding faulty concepts or misleading biases into the training data that are not present in reality (66,67). Any uncertainty or bias introduced during the training of the synthetic data generator is inherent in the resulting samples.…”

Accelerated design ofEscherichia coligenomes with reduced size using a whole-cell model and machine learning

“…The process for training such a machine learning model involves running the mechanistic model several times in order to generate the training and test sets. After that, the ML model is trained using this data, and once the accuracy is satisfactory, it can be used to replace the original mechanistic model in future simulations [14]. Some common techniques used to create surrogate models include regression models [19,43], Gaussian processes [21], support vector machines [43], decision tree-based models [16,17] and neural networks [17,15].…”

“…ML surrogates, or emulators as they are sometimes known, are black-box models trained to approximate the behaviour of complex and computationally expensive mathematical models [14]. The major advantage of their use is the reduced computational resources required, thus allowing the simulation time to fall from hours or days to seconds.…”