Credible practice of modeling and simulation in healthcare: ten rules from a multidisciplinary perspective

Abstract

The complexities of modern biomedicine are rapidly increasing. Thus, modeling and simulation have become increasingly important as a strategy to understand and predict the trajectory of pathophysiology, disease genesis, and disease spread in support of clinical and policy decisions. In such cases, inappropriate or ill-placed trust in the model and simulation outcomes may result in negative outcomes, and hence illustrate the need to formalize the execution and communication of modeling and simulation practices. Although verification and validation have been generally accepted as significant components of a model’s credibility, they cannot be assumed to equate to a holistic credible practice, which includes activities that can impact comprehension and in-depth examination inherent in the development and reuse of the models. For the past several years, the Committee on Credible Practice of Modeling and Simulation in Healthcare, an interdisciplinary group seeded from a U.S. interagency initiative, has worked to codify best practices. Here, we provide Ten Rules for credible practice of modeling and simulation in healthcare developed from a comparative analysis by the Committee’s multidisciplinary membership, followed by a large stakeholder community survey. These rules establish a unified conceptual framework for modeling and simulation design, implementation, evaluation, dissemination and usage across the modeling and simulation life-cycle. While biomedical science and clinical care domains have somewhat different requirements and expectations for credible practice, our study converged on rules that would be useful across a broad swath of model types. In brief, the rules are: (1) Define context clearly. (2) Use contextually appropriate data. (3) Evaluate within context. (4) List limitations explicitly. (5) Use version control. (6) Document appropriately. (7) Disseminate broadly. (8) Get independent reviews. (9) Test competing implementations. (10) Conform to standards. Although some of these are common sense guidelines, we have found that many are often missed or misconstrued, even by seasoned practitioners. Computational models are already widely used in basic science to generate new biomedical knowledge. As they penetrate clinical care and healthcare policy, contributing to personalized and precision medicine, clinical safety will require established guidelines for the credible practice of modeling and simulation in healthcare.

A theoretical model of health management using data-driven decision-making: the future of precision medicine and health

“…The presented model of HT management can be broadly applied. In the era of precision medicine and health, it is crucial to identify critical factors that significantly increase or reduce health risk(s) in all branches of medicine [10,31]. The essential task, not just in orthopaedics, is to identify the best-suited therapy for an individual patient, as well as to minimise the harm associated with a particular intervention, because even well-established therapeutic interventions have been questioned in the last few decades [14,32].…”

The SPARC DRC: Building a resource for the autonomic nervous system community

“…Computational services must have associated meta-data that documents the required inputs (type, format, etc.) and produced outputs (potentially even their physiological meaning), the hardware requirements, the degree of reliability (e.g., degree of verification, validation, and certification, suitable context-of-use - the quality assurance information is organized according to the ‘Ten Simple Rules’ (Erdemir et al, 2020)), etc. An online form is available for submitting the required information as part of the semi-automatic service-creation-process.…”

“…codeclimate and dependabot ), unit-, regression-, and end-to-end-testing (e.g., with puppeteer ), multi-staged release (development, staging, and production versions), while service QA is established providing a template mechanism ( cookiecutter ) that includes automatic testing of communication interfaces, support for the co-submission of test benchmarks. Supporting information on verification and validation can be attached to services and studies as part of displayed meta-data (complementing the compulsory meta-data from the cMIS), which is part of the existing and forthcoming functionality to facilitate, encourage, and enforce adoption of the ‘Ten Simple Rules’ (Erdemir et al, 2020).…”

Fluid–Structure Interaction Analyses of Biological Systems Using Smoothed-Particle Hydrodynamics

“…This finding demonstrates not only the importance of proper validation but also the necessity of having such validation verified by independent reviews. A committee on the credible practice of modeling and simulation in healthcare, an interdisciplinary group seeded from a U.S. inter-agency initiative, has recently codified best practices in order to formalize the execution and communication of modeling and simulation practices in healthcare [76]. They suggest ten rules for the credible practice of modeling and simulation in healthcare stated as follows: (1) Define context clearly.…”