Data assimilation and agent-based modelling: towards the incorporation of categorical agent parameters

Abstract

This paper explores the use of a particle filter—a data assimilation method—to incorporate real-time data into an agent-based model. We apply the method to a simulation of real pedestrians moving through the concourse of Grand Central Terminal in New York City (USA). The results show that the particle filter does not perform well due to (i) the unpredictable behaviour of some pedestrians and (ii) because the filter does not optimise the categorical agent parameters that are characteristic of this type of model. This problem only arises because the experiments use real-world pedestrian movement data, rather than simulated, hypothetical data, as is more common. We point to a potential solution that involves resampling some of the variables in a particle, such as the locations of the agents in space, but keeps other variables such as the agents’ choice of destination. This research illustrates the importance of including real-world data and provides a proof of concept for the application of an improved particle filter to an agent-based model. The obstacles and solutions discussed have important implications for future work that is focused on building large-scale real-time agent-based models.

An Agent-Based Model of the 2020 International Policy Diffusion in Response to the COVID-19 Pandemic with Particle Filter

“…The DA approach we choose is a particle filter since it is well-suited for highly non-linear systems and is able to cope with categorical variables; a feature our model relies on. Moreover it has been applied to a few ABMs already, although they were mostly simulating pedestrian dynamics (Wang & Hu 2015;Malleson et al 2020;Ternes et al 2022). Data-assimilation with a diffusion-oriented ABM let alone a policy focus is a novel application.…”

“…Examples of the use of agent-based models with data assimilation are extremely rare. Only relatively recent literature applies particle filters (Wang & Hu 2015;Ternes et al 2022;Malleson et al 2020;Hu 2022;Lueck et al 2019), other sequential Monte-Carlo sampling techniques (Tang & Malleson 2022) and varieties of the Kalman Filter (Clay et al 2021;Ward et al 2016) to ABMs, typically for crowd simulation or more general population movement.…”

“…In ABMs, agents might possess a diverse set of behavioural parameters including numeric and categorical ones. Data assimilation algorithms however have been developed to optimise continuous numerical variables in weather and climate modelling (Ternes et al 2022). ABMs are essentially discrete event simulations (Delli Gatti et al 2018).…”

“…For example, Malleson et al (2020) found, in their model of pedestrians, that data assimilation with a particle filter became extremely computationally costly because, as the number of agents in the environment increases, exponentially more particles are required to achieve a constant (low) error rate. In addition, Ternes et al (2022) found difficulties using a particle filter in an agent-based crowd simulation because particle deprivation meant that the algorithm was not able to adequately search the space of possible model trajectories and ruled out models that would ultimately have predicted the system well. Ternes et al (2022) proposed a resolution to this problem by filtering some parts of the model state space but not others.…”

“…In addition, Ternes et al (2022) found difficulties using a particle filter in an agent-based crowd simulation because particle deprivation meant that the algorithm was not able to adequately search the space of possible model trajectories and ruled out models that would ultimately have predicted the system well. Ternes et al (2022) proposed a resolution to this problem by filtering some parts of the model state space but not others. Variations of the Kalman Filter are potentially more efficient than the particle filter, but cannot estimate categorical parameters, although Clay et al (2021) propose a 'reversible jump' mechanism to get round this drawback.…”