Automatic Reuse, Adaption, and Execution of Simulation Experiments via Provenance Patterns

Abstract

Simulation experiments are typically conducted repeatedly during the model development process, for example, to re-validate if a behavioral property still holds after several model changes. Approaches for automatically reusing and generating simulation experiments can support modelers in conducting simulation studies in a more systematic and effective manner. They rely on explicit experiment specifications and, so far, on user interaction for initiating the reuse. Thereby, they are constrained to support the reuse of simulation experiments in a specific setting. Our approach now goes one step further by automatically identifying and adapting the experiments to be reused for a variety of scenarios. To achieve this, we exploit provenance graphs of simulation studies, which provide valuable information about the previous modeling and experimenting activities, and contain meta-information about the different entities that were used or produced during the simulation study. We define provenance patterns and associate them with a semantics, which allows us to interpret the different activities, and construct transformation rules for provenance graphs. Our approach is implemented in a Reuse and Adapt framework for Simulation Experiments (RASE) which can interface with various modeling and simulation tools. In the case studies, we demonstrate the utility of our framework for a) the repeated sensitivity analysis of an agent-based model of migration routes, and b) the cross-validation of two models of a cell signaling pathway.

Simulation Studies of Social Systems – Telling the Story Based on Provenance Patterns

“…For the modeling and model analysis, the central part of a simulation study (see Figure 1), important entities and processes have been identified in the literature [8,10]. Building on this, Wilsdorf et al [18] identified provenance patterns for model building and simulation experiments arising in a simulation study: certain activities within a simulation study will always use and produce certain types of entities. A pattern consists of an activity at its center, and the types of entities that are used and produced by this activity.…”

“…In this work, we extend the method of provenance patterns to consider the data-related processes. In the following, we summarise the entities and activities and the arising patterns in modeling and model analysis, as defined in [18]. Building on that, we then extend the scope of the approach by identifying entities, activities and patterns for primary and secondary data collection.…”

“…Wilsdorf et al [18] mapped the considerations of existing reporting guidelines (e.g., [2,28]) into the provenance standard W3C PROV, by identifying the central activities of a simulation study. As in [18], we distinguish the following entities in the modeling and analysis part of the study; Research Question (RQ), Simulation Model (SM), Simulation Experiment Specification (SE), Simulation Data (SD), Requirement (R), Assumption (A) and Other (O).…”

“…Wilsdorf et al [18] mapped the considerations of existing reporting guidelines (e.g., [2,28]) into the provenance standard W3C PROV, by identifying the central activities of a simulation study. As in [18], we distinguish the following entities in the modeling and analysis part of the study; Research Question (RQ), Simulation Model (SM), Simulation Experiment Specification (SE), Simulation Data (SD), Requirement (R), Assumption (A) and Other (O). In the patterns, we also make use of an entity type Data (D), as a general placeholder for any kind of data, including simulation data produced in an experiment, but also primary and secondary data (which we otherwise distinguish due to different documentation requirements).…”

“…To systematically and accessibly document entire social simulation studies, which include extensive data collection, evaluation, analysis, and adaptation, in this paper we present an approach based on provenance patterns specified in the W3C PROV standard. In [18], provenance patterns have been identified for the documentation of and reasoning about central activities of simulation studies, such as model building and refinement, or conducting simulation experiments for verification, calibration, and validation. These patterns shall now be extended to capture data evaluation schemes used to assess the quality and uncertainty of secondary data sources, and to the collection of primary data, exemplified by psychological experiments or interviews conducted to support the agent-based modeling of human decision processes.…”