Synergistic Integration Between Machine Learning and Agent-Based Modeling: A Multidisciplinary Review

Abstract

Agent-based modeling (ABM) involves developing models in which agents make adaptive decisions in a changing environment. Machine-learning (ML) based inference models can improve sequential decision-making by learning agents' behavioral patterns. With the aid of ML, this emerging area can extend traditional agent-based schemes that hardcode agents' behavioral rules into an adaptive model. Even though there are plenty of studies that apply ML in ABMs, the generalized applicable scenarios, frameworks, and procedures for implementations are not well addressed. In this article, we provide a comprehensive review of applying ML in ABM based on four major scenarios, i.e., microagent-level situational awareness learning, microagent-level behavior intervention, macro-ABM-level emulator, and sequential decision-making. For these four scenarios, the related algorithms, frameworks, procedures of implementations, and multidisciplinary applications are thoroughly investigated. We also discuss how ML can improve prediction in ABMs by trading off the variance and bias and how ML can improve the sequential decision-making of microagent and macrolevel policymakers via a mechanism of reinforced behavioral intervention. At the end of this article, future perspectives of applying ML in ABMs are discussed with respect to data acquisition and quality issues, the possible solution of solving the convergence problem of reinforcement learning, interpretable ML applications, and bounded rationality of ABM.

Using Machine Learning for Agent Specifications in Agent-Based Models and Simulations: A Critical Review and Guidelines

“…5 There are already a handful of literature reviews on using ML in ABMS. Zhang et al (2021) review literature on ML for the agents' decision-making, distinguishing between micro-agent-level situational awareness learning, micro-agent-level behaviour intervention, macro-ABMS-level emulator, and sequential decision-making. Dahlke et al (2020) provide a general literature review on using ML for the structural specifications and outputs of ABMS.…”

Innovations in integrating machine learning and agent-based modeling of biomedical systems

“…Several prior reviews have described how ML can be leveraged in computational modeling, e.g., by Alber et al (2019) and by Peng et al (2021). In addition, the idea of synergistically integrating ML and ABM (Figure 2) has existed since at least Rand's (2006) early report, and includes more recent works such as by Giabbanelli (2019), Brearcliffe and Crooks (2021), and Zhang et al (2021). The remainder of our present Review focuses more on the utility of ML within ABMs, and attempts to offer some guiding principles on how and when these integrations are feasible in simulating different scales of biology.…”

“…As an example of how to interpret this diagram, note that mode ②, "behavior intervention", entails application of "online" ML methods to modify agent behavior/action policies, which is essentially reinforcement learning. Note that this illustration is adapted from one that appears in Zhang et al (2021), where further details may be found.…”

Exploring the potential impacts of machine learning on trust in fishery management

“…For example, ML provides methods to extend agent based modelling and participatory modelling into adaptive models where agents' behaviours are hardcoded. This allows fishery managers and stakeholders to explore model scenarios, such as behaviour intervention, sequential decision-making, and situational awareness learning (Zhang et al, 2021). Researchers have explored the use of reinforcement learning based methods to address IUU fishing.…”