Multi-Model and Network Inference Based on Ensemble Estimates: Avoiding the Madness of Crowds

Abstract

The development of mathematical models of biological systems has largely relied on a mix of biological intuition, mathematical expediency, and comparisons with data. Models are hard to develop and hard to validate. Recent progress in theoretical systems biology, applied mathematics and computational statistics has, however, led to a proliferation of models and methods that allow us to compare quantitatively the performance of different candidate models at describing a particular biological system or process. Model selection has been applied with great success to problems where a small number -typically less than 10 -of models are compared, but recently studies have started to consider thousands and even millions of candidate models. As formal computational and automated systems increasingly aim to replace or augment biological intuition this trend is likely to continue. Often no single model is chosen as the best, but a large number of models is found to be compatible with data, and these are then often studied as an ensemble. In other situations several prediction algorithms are used to make ensemble predictions, for example, about the presence or absence of molecular interactions. Using a set of test-cases I outline when such ensemble estimators can be relied upon, and when they are likely to fail. If model or predictor ensembles are not carefully constructed and curated, predictions and estimates based on ensembles can perform worse than good individual predictors.

Gaining confidence in inferred networks

“…knowing which inferred networks are worth further consideration, and which ones are best ignored will have a profound impact on our ability to make use of networks. Quickly being able to reject some network inferences does allow for more streamlined analysis, but is also essential [43] if we want to base predictions on ensembles of network inference methods.…”