Automatic Reuse, Adaption, and Execution of Simulation Experiments via Provenance Patterns

Abstract

Simulation experiments are typically conducted repeatedly during the model development process, for example, to re-validate if a behavioral property still holds after several model changes. Approaches for automatically reusing and generating simulation experiments can support modelers in conducting simulation studies in a more systematic and effective manner. They rely on explicit experiment specifications and, so far, on user interaction for initiating the reuse. Thereby, they are constrained to support the reuse of simulation experiments in a specific setting. Our approach now goes one step further by automatically identifying and adapting the experiments to be reused for a variety of scenarios. To achieve this, we exploit provenance graphs of simulation studies, which provide valuable information about the previous modeling and experimenting activities, and contain meta-information about the different entities that were used or produced during the simulation study. We define provenance patterns and associate them with a semantics, which allows us to interpret the different activities, and construct transformation rules for provenance graphs. Our approach is implemented in a Reuse and Adapt framework for Simulation Experiments (RASE) which can interface with various modeling and simulation tools. In the case studies, we demonstrate the utility of our framework for a) the repeated sensitivity analysis of an agent-based model of migration routes, and b) the cross-validation of two models of a cell signaling pathway.

Simulation Studies of Social Systems – Telling the Story Based on Provenance Patterns

“…This benefit is not restricted to the consumer of the documentation: Wilsdorf at al. [17] demonstrate how the provenance graph can be used to automatically generate new versions of simulations experiments for new model versions. The provenance graph can be automatically generated during the simulation study, e.g., by exploiting workflow systems to conduct the simulation study [10].…”

“…Wilsdorf et al [17] mapped those considerations into the provenance standard W3C PROV, by identifying entities such as research question, simulation model, simulation experiment (specification), simulation data, data, requirement, qualitative model, and assumptions, and related those entities by activities such as creating simulation model, refining simulation model, reimplementing simulation model, calibrating, analyzing, and validating simulation model.…”

“…1), important entities and processes have been identified in the literature [7], [9]. Building on this, Wilsdorf et al [17] identified provenance patterns arising in a simulation study: certain activities within a simulation study will always use and produce certain types of entities. A pattern consists of an activity at its center, and the types of entities that are used and produced by this activity.…”

“…Compared to Wilsdorf et al [17], we added one additional pattern Identifying Research Question (h). Newly identified research questions are often a major driver of long term simulation studies -as well as important results.…”

“…To systematically and accessibly document entire demographic simulation studies, which also include extensive data evaluation, analysis, adaptations and their collection, we will present an approach based on provenance patterns specified in the W3C PROV standard. In [17], provenance patterns have already been identified for the documentation of and reasoning about central activities of simulation studies such as model building, refinement, verification, calibration, and validation. These patterns shall now be extended to capture data evaluation schemes to assess the quality or uncertainty of data sources, and the conducting of psychological experiments or interviews to support the agent-based modeling of human decision processes.…”