The Use of Surrogate Models to Analyse Agent-Based Models

Abstract

The utility of Agent Based Models (ABMs) for decision making support as well as for scientific applications can be increased considerably by the availability and use of methodologies for thorough model behaviour analysis. In view of their intrinsic construction, ABMs have to be analysed numerically. Furthermore, ABM behaviour is o en complex, featuring strong non-linearities, tipping points, and adaptation. This easily leads to high computational costs, presenting a serious practical limitation. Model developers and users alike would benefit from methodologies that can explore large parts of parameter space at limited computational costs. In this paper we present a methodology that makes this possible. The essence of our approach is to develop a cost-e ective surrogate model based on ABM output using machine learning to approximate ABM simulation data. The development consists of two steps, both with iterative loops of training and cross-validation. In the first part, a Support Vector Machine (SVM) is developed to split behaviour space into regions of qualitatively di erent behaviour. In the second part, a Support Vector Regression (SVR) is developed to cover the quantitative behaviour within these regions. Finally, sensitivity indices are calculated to rank the importance of parameters for describing the boundaries between regions, and for the quantitative dynamics within regions. The methodology is demonstrated in three case studies, a di erential equation model of predator-prey interaction, a common-pool resource ABM and an ABM representing the Philippine tuna fishery. In all cases, the model and the corresponding surrogate model show a good match. Furthermore, di erent parameters are shown to influence the quantitative outcomes, compared to those that influence the underlying qualitative behaviour. Thus, the method helps to distinguish which parameters determine the boundaries in parameter space between regions that are separated by tipping points, or by any criterion of interest to the user.

Using Machine Learning for Agent Specifications in Agent-Based Models and Simulations: A Critical Review and Guidelines

“…Often, increasing computational efficiency appears a secondary challenge next to improving the accuracy of behavioural modelling. In the remaining cases, ML is often used to create surrogate or metamodels which learn the relationship between ABMS in-and output to achieve reliable results in a more efficient manner, see Vahdati et al (2019), ten Broeke et al (2021, or Yousefi et al (2018). As outlined by van der Hoog (2019, p. 1260) surrogate ML models have the potential to drastically reduce "the complexity and computational load of simulating agent-based models".…”

Assessing the quality of land system models: moving from valibration to evaludation

“…To the extent that empirical ABMs are more complicated than other land system models, they are disproportionately harder to calibrate and evaluate because small increases in complicatedness (in the sense of having more components; Sun et al (2016)) require large increases in supporting data (Srikrishnan & Keller, 2021). These difficulties may be overcome through the use of theoretical 'priors' (Taghikhah et al, 2021) or techniques such as surrogate modelling or Machine Learning to reduce computational costs (Storm et al, 2020;ten Broeke et al, 2021), but are still likely to represent substantial challenges.…”

“…A number of robust statistical approaches have been suggested to improve on current practice (Saltelli et al, 2019;Stepanyan et al, 2021). Particularly promising are techniques using machine learning, for instance to create surrogates of ABMs that can be analysed far more quickly than the model itself (Angione et al, 2022;ten Broeke et al, 2021).…”

Pattern‐oriented analysis of system dynamics models via random forests

“…In early studies, linear regression techniques were widely used to construct metamodels (Blanning, 1975; Friedman and Friedman, 1985; Kleijnen, 1975, 1979). Recent advancements in the machine‐learning domain have enabled analysts to use different techniques, such as neural networks (Can and Heavey, 2012; Sabuncuoglu and Touhami, 2002; Sharifnia et al ., 2021), random forests (RFs) (Edali and Yücel, 2019, 2020; Stolfi and Castiglione, 2021), Gaussian processes (Betancourt et al ., 2020; Dosi et al ., 2018), support vector machines (Edali and Yücel, 2018; Ten Broeke et al ., 2021), and radial basis functions (Jakobsson et al ., 2010; Mullur and Messac, 2006). In addition, comparative analyses of some of these techniques based on their accuracy, interpretability, robustness, and efficiency have been presented in some studies.…”