Active learning metamodeling for policy analysis: Application to an emergency medical service simulator

Abstract

Simulation approaches constitute a well-established tool to model, understand and predict the behavior of transportation systems, and ultimately to assess the performance of transportation policies. Due to their multidimensionality nature, such systems are not often approachable through conventional analytic methods, making simulation modeling the only reliable tool of study. Nevertheless, simulation models can turn out to be computationally expensive when embedded with enough detail. An immediate answer to this shortcoming is the use of simulation metamodels that are designed to approximate the simulators' results.In this work, the authors propose a metamodeling approach based on active learning that seeks to improve the exploration process of the simulation input space and the associated output behavior. A Gaussian Process (GP) is used as a metamodel to approximate the function inherently defined by the simulation model itself. The GPs are able to nicely handle the uncertainty associated with their predictions, which eventually can be improved with active learning through simulation runs. This provides a practical and efficient way to analyze the simulator's behavior and therefore to assess the performance of policies regarding the underlying real-world systems and services.The authors illustrate the proposed methodology using an Emergency Medical Service (EMS) simulator. Two outputs are analyzed and compared, namely, the survival rate and response time averages. The medical emergency response time recommendation of eight minutes is explored as well its relation with the survival rate. The results show that this methodology is able to identify regions in the simulation input space that might affect the performance and success of medical policies with regards to emergency vehicle dispatching services.

A review on simulation-based metamodeling in emergency healthcare: methodology, applications, and future challenges

“…They used a powerful machine than the learning (ML) tool, active learning (AL) in a SBM approach, and used predictive variance (PV) to incorporate noise and guide metamodel fitting instead of using a predefined database. 25 Antunes et al 56 developed Antunes et al 7 approach by integrating user experience. In contrast to the trial and error method, the proposed method extracts as much information as possible from a few data points using an algorithm with a directional search toward inputs, leading to outputs close to a predefined value.…”

“…5,6 On the other hand, it is a descriptive technique used to model and analyze the systems under study by testing different scenarios without any proof of their optimality. 7 Given that exploring the entire search space in real-life optimization problems and evaluating all possible scenarios are computationally expensive, simulation often remains impractical for large-scale problems.…”

Simulation Metamodeling Approach to Complex Design of Garment Assembly Lines

“…The term indicates a mathematical approximation that models the behavior of another model [8,12]. The objective of metamodeling is to reduce the computational cost of the simulation model during the optimization process [9,14].…”

“…For instance, regression techniques of metamodeling have been used to estimate relations between model inputs and outputs based on results of a probabilistic sensitivity analysis [10][11][12]. More specifically, metamodels build a closedform mathematical expression to approximate the input and output relationship implied by the simulation model based on simulation experiments runs at selected design points in advance [13,14]. This can be easily evaluated in a spreadsheet environment "on demand" to answer what-if questions without needing to run lengthy simulations [15].…”

Simulation metamodeling approach to complex design of garment assembly lines

“…For instance, regression techniques of metamodeling have been used to estimate relationships between model inputs and outputs based on results of a probabilistic sensitivity analysis [ 21 – 23 ]. More specifically, metamodels build a closed-form mathematical expression to approximate the input and output relationship implied by the simulation model based on simulation experimental runs at selected design points in advance [ 24 , 25 ]. This can be easily evaluated in a spreadsheet environment “on demand” to answer what-if questions without the need to run lengthy simulations [ 26 ].…”