Machine Learning to Derive Complex Behaviour in Agent-Based Modellzing

Abstract

The use of machine learning algorithms to enrich agent-based models has increased over the past years. This integration adds value when combining the advantages of the data-driven approach and the possibilities to explore future situations and human interventions. However, this integrating is still in its infant stage. Full integration of learning algorithms and agent-based models is often technically challenging and can make the behavioural rules of the agents less transparent. Experiments are needed in which different integration strategies are compared using the same agent-based model to determine when each of these approaches is most effective. In this paper, we present a comparison of two versions of the same cholera model. In the initial version, agent behaviour was driven directly by a learning algorithm. In our experiments, we replace this strategy by applying a learning algorithm directly on the data and implement the outcomes as behaviour rules in the model. The results showed that when the integration aims to create agents that show characteristics that are data-driven, deriving rules based on these data is a good alternative. In addition, a key element in this strategy is the dataset. A large dataset representing the behaviour of different types of agents over the complete time period is needed.

Innovations in integrating machine learning and agent-based modeling of biomedical systems

“…The decision tree scheme differs from the BN approach in that the decision tree does not require expert opinion or sequential learning, and instead derives (novel) agent rules from scratch by determining a tree-like model/path of how each agent considers the predictor variables to arrive at a decision regarding usage of river water. The decision tree-based approach yielded ABM predictions with different numbers of infected individuals (Augustijn et al, 2020). This discrepancy could be anticipated because, as outlined in Figure 4, the two different ML integrations led to two fundamentally different rulesets, thus affecting the emergent properties/outcomes of the system.…”

“…These environmental variables form the input for an ML algorithm that outputs a decision for the agent to enact. The left example refers to an ABM developed to simulate cholera spread in Kumasi, Ghana, wherein supervised learning algorithms trained on survey data were used to select the most probable behavior based on environmental variables (Abdulkareem et al, 2019;Augustijn et al, 2020). Several other epidemiological ABMs have leveraged large datasets to train supervised learning algorithms to determine agent behavior (Day et al, 2013;Abdulkareem et al, 2019;Alexander Jr et al, 2019;Augustijn et al, 2020).…”

“…The left example refers to an ABM developed to simulate cholera spread in Kumasi, Ghana, wherein supervised learning algorithms trained on survey data were used to select the most probable behavior based on environmental variables (Abdulkareem et al, 2019;Augustijn et al, 2020). Several other epidemiological ABMs have leveraged large datasets to train supervised learning algorithms to determine agent behavior (Day et al, 2013;Abdulkareem et al, 2019;Alexander Jr et al, 2019;Augustijn et al, 2020). The right-hand example references multicellular ABMs that simulate individual cell behaviors in a tissue, with cellular "decisions" being made based on either Q-learning (Zangooei and Habibi, 2017) or ANN approaches (Gerlee andAnderson, 2007, 2008;Kazmi et al, 2012a;Kazmi et al, 2012b;Al-Mamun et al, 2013;Al-Mamun et al, 2014;Al-Mamun et al, 2016;Abdulkareem et al, 2019).…”

“…This study shows not only that Bayesian networks are a viable learning algorithm to incorporate expert opinion into ABMs when the available training data is limited, but also that the feedback process in sequential learning can further improve the accuracy of a learning algorithm by utilizing data generated by the ABM simulation. In another study, Augustijn et al (2020) took an alternative approach to determining human agent water use in the aforementioned cholera-spread ABM: that work trained decision trees on the same training dataset as the earlier study (Abdulkareem et al, 2019) in order to determine whether an agent will use river water based on the same predictor variables. The decision tree scheme differs from the BN approach in that the decision tree does not require expert opinion or sequential learning, and instead derives (novel) agent rules from scratch by determining a tree-like model/path of how each agent considers the predictor variables to arrive at a decision regarding usage of river water.…”

Machine Learning based predictive modelling and simulation of network failure events

“…More recent works like [99], propose using agent-based simulation for the generation of training sets for neural networks. In [100], decision trees are used in the agents to carry out simulations on expanding the cholera contagions. [101] proposes using algorithms that allow agents to perform reinforced learning based on the data they observe during the simulation.…”