Using Machine Learning for Agent Specifications in Agent-Based Models and Simulations: A Critical Review and Guidelines

Abstract

Agent-based modelling and simulation (ABMS), whether simple toy models or complex data-driven ones, is regularly applied in various domains to study the system-level patterns arising from individual behaviour and interactions. However, ABMS still faces diverse challenges such as modelling more representative agents or improving computational efficiency. Research shows that machine learning (ML) techniques, when used in ABMS can address such challenges. Yet, the ABMS literature is still marginally leveraging the benefits of ML. One reason is the vastness of the ML domain, which makes it difficult to choose the appropriate ML technique to overcome a specific modelling challenge. This paper aims to bring ML more within reach of the ABMS community. We first conduct a structured literature review to investigate how the ABMS process uses ML techniques. We focus specifically on articles where ML is applied for the structural specifications of models such as agent decision-making and behaviour, rather than just for analysing output data. Given that modelling challenges are mainly linked to the purpose a model aims to serve (e.g., behavioural accuracy is required for predictive models), we frame our analysis within different modelling purposes. Our results show that Reinforcement Learning algorithms may increase the accuracy of behavioural modelling. Moreover, Decision Trees, and Bayesian Networks are common techniques for data pre-processing of agent behaviour. Based on the literature review results, we propose guidelines for purposefully integrating ML in ABMS. We conclude that ML techniques are specifically fit for currently underrepresented modelling purposes of social learning and illustration; they can be used in a transparent and interpretable manner.

Examining the Interplay Between National Strategies and Value Change in the Battle Against COVID-19: An Agent-Based Modelling Inquiry

“…Using ML to train agents in the model 3.8 ML techniques refer to algorithms that have the ability to find patterns and predict outcomes by learning from input data and without programming all requirements explicitly (Samuel 1959;Murphy 2012). ML techniques can provide great potential to bring higher degrees of intelligence and learning into the models (Macal & North 2010;Rand & Rust 2011;Kavak et al 2018;Dehkordi et al 2023).…”

Editorial: Modelling Values in Social, Technical, and Ecological Systems

“…Psychological studies suggest that values are typically formed before reaching adulthood and may change only in case of significant crises (Steinert, 2021, e.g., Covid-19) and "life-changing" events like moving abroad (Bardi et al, 2014;Sagiv et al, 2017). Due to extensive empirical grounding, psychological theories of values are the most popular among agentbased modellers (e.g., Gore et al, 2019;Boshuijzen-van Burken et al, 2020;Beheshti & Sukthankar, 2014;Kreulen et al, 2022), including contributors to this special section (e.g., Czupryna et al, 2024;Ale Ebrahim Dehkordi et al, 2024;Davis et al, 2024;Jager et al, 2024;Shults et al, 2024). However, psychology typically takes an individual perspective or aggregated individual perspectives on values and so does not study what might happen in more social settings.…”

Systematic Review of Agent-Based and System Dynamics Models for Social-Ecological System Case Studies

“…Finally, implementing machine learning (ML) algorithms in the ABM and SD can enhance their performance in modeling SES. In ABMs, ML algorithms can be used to develop more sophisticated agent behaviors and decision-making rules, whereas agents can learn from their interactions with the environment and other agents, allowing for the representation of adaptive and evolving behaviors [90]. In SD, ML algorithms can be used to optimize model parameters, which can be particularly valuable in scenarios where finding the best parameter values is challenging [91].…”