Credibility, Replicability, and Reproducibility in Simulation for Biomedicine and Clinical Applications in Neuroscience

Abstract

Modeling and simulation in computational neuroscience is currently a research enterprise to better understand neural systems. It is not yet directly applicable to the problems of patients with brain disease. To be used for clinical applications, there must not only be considerable progress in the field but also a concerted effort to use best practices in order to demonstrate model credibility to regulatory bodies, to clinics and hospitals, to doctors, and to patients. In doing this for neuroscience, we can learn lessons from long-standing practices in other areas of simulation (aircraft, computer chips), from software engineering, and from other biomedical disciplines. In this manuscript, we introduce some basic concepts that will be important in the development of credible clinical neuroscience models: reproducibility and replicability; verification and validation; model configuration; and procedures and processes for credible mechanistic multiscale modeling. We also discuss how garnering strong community involvement can promote model credibility. Finally, in addition to direct usage with patients, we note the potential for simulation usage in the area of Simulation-Based Medical Education, an area which to date has been primarily reliant on physical models (mannequins) and scenario-based simulations rather than on numerical simulations.

Characteristics of mathematical modeling languages that facilitate model reuse in systems biology: A software engineering perspective

“…References: There is a consensus that multi-scale modeling requires some form of modularity for hierarchical composition [36,[40][41][42][43][44]. More specifically, Hellerstein et al [35] and Mulugeta et al [25] both suggest that object-oriented programming might be an especially promising way to implement modularity.…”

“…It provides an environment for graphical block diagrams called Simulink (https: //www.mathworks.com/products/simulink.html) and a declarative language for designing physical systems called Simscape (https://www.mathworks.com/ products/simscape.html). While MATLAB is still popular [24,25], the opensource programming language Python also gains increasing interest in the com-munity [24,[26][27][28][29]. Usually models are not built in Python itself, but researchers have created packages such as PySB [28] and the python simulator for cellular systems (PySCes) [26] that define embedded domain-specific languages (DSLs) which facilitate the creation of mathematical models for specific use cases.…”

A Reproducible Neurobiology of Depressive Rumination

“…To test and explore these hypotheses, we utilize data from two neuroimaging modalities, dMRI and rsfMRI. Given the absolute importance of replication in cognitive neuroscience (Button et al, 2013) and the growing need for more credible and reproducible neurobiological models of disordered cognition (Mulugeta et al, 2018), we additionally perform our analyses across two separate datasets, acquired independently and at different locations. This approach, which is rarely used in social or biological sciences (Button et al, 2013;Poldrack et al, 2017), would allow us to determine whether initial discovery findings could be directly replicated in an independent sample.…”

NetPyNE: a tool for data-driven multiscale modeling of brain circuits

“…Other key elements of the modeling workflow such as ensuring replicability, optimizing parameters and analyzing results also need to be implemented separately by each user. 10,11 Lack of model standardization makes it hard to understand, reproduce and reuse many existing models and simulation results.…”