Reporting guidelines for simulation‐based research in social sciences

Abstract

Reproducibility of research is critical for the healthy growth and accumulation of reliable knowledge, and simulation-based research is no exception. However, studies show many simulation-based studies in the social sciences are not reproducible. Better standards for documenting simulation models and reporting results are needed to enhance the reproducibility of simulation-based research in the social sciences. We provide an initial set of Reporting Guidelines for Simulation-based Research (RGSR) in the social sciences, with a focus on common scenarios in system dynamics research. We discuss these guidelines separately for reporting models, reporting simulation experiments, and reporting optimization results. The guidelines are further divided into minimum and preferred requirements, distinguishing between factors that are indispensable for reproduction of research and those that enhance transparency. We also provide a few guidelines for improved visualization of research to reduce the costs of reproduction. Suggestions for enhancing the adoption of these guidelines are discussed at the end.Acknowledgments: We would like to thank Mohammad Jalali for providing excellent research assistance for this paper. We thank David Lane for his helpful suggestions.

The Challenge

“…In addition, such business model analyses could be extended with quantitative simulation models for estimation in the context of business model innovation (Rahmandad and Sterman 2012;Groesser and Jovy 2016). Simulating business cases in a systematic and reliable manner would allow for informed decisions to be made on which upgrades should be conducted.…”

Raising awareness in model-based energy scenario studies—a transparency checklist

“…In the context of model-based research in the social sciences, Rahmandad and Sterman [32] provide reporting guidelines to facilitate model reproducibility. They distinguish between a 'minimum' and a 'preferred' reporting requirement.…”

“…Later, the authors assess the critical points raised against ODD and offer an updated and improved version of the protocol [34]. Concerning the SD approach, Rahmandad and Sterman [32] illustrate how to implement their reporting guidelines using an innovation diffusion model. In principle, the SD modeling approach is suitable for a high level of model transparency.…”

Information diffusion through social networks: The case of an online petition

“…The equations for these subsystems are presented in Section 3.3. We fully documented the model following a set of minimum reporting requirements (Rahmandad & Sterman, 2012 This subsystem includes the process through which the unaware target population moves to the pool of people with a pending message. The unaware target population ( ) receives announcements via: 1) email or a shared posts on Facebook (initial announcement rate, ) from the core group; and 2) through the sharing of the petition (rate of sharing, ) by those who have already signed the petition.…”