Towards systematic, data-driven validation of a collaborative, multi-scale model of Caenorhabditis elegans

Abstract

The OpenWorm Project is an international open-source collaboration to create a multi-scale model of the organism Caenorhabditis elegans. At each scale, including subcellular, cellular, network and behaviour, this project employs one or more computational models that aim to recapitulate the corresponding biological system at that scale. This requires that the simulated behaviour of each model be compared with experimental data both as the model is continuously refined and as new experimental data become available. Here we report the use of SciUnit, a software framework for model validation, to attempt to achieve these goals. During project development, each model is continuously subjected to data-driven 'unit tests' that quantitatively summarize model-data agreement, identifying modelling progress and highlighting particular aspects of each model that fail to adequately reproduce known features of the biological organism and its components. This workflow is publicly visible via both GitHub and a web application and accepts community contributions to ensure that modelling goals are transparent and well-informed. This article is part of a discussion meeting issue 'Connectome to behaviour: modelling C. elegans at cellular resolution'.

The Virtual Patch Clamp: Imputing C. elegans Membrane Potentials from Calcium Imaging

“…Previous work has investigated performing imputation of neural spikes, membrane potentials, calcium dynamics, model parameters and connectivities [37,38,39,40,41,42]. However these studies do not operate under a biologically accurate model of the dynamics of a whole connectome, instead investigating individual or small networks of fully observed synthetic neurons.…”

“…To conclude we note that in the past year several articles discussing open research issues pertaining to C. elegans simulation have been produced by the C. elegans community [4,42,47,48]. Figure 1(d) outlines the community planned development pipeline for C. elegans simulation.…”

OpenWorm: overview and recent advances in integrative biological simulation of Caenorhabditis elegans

“…In contrast to simple unit tests, which verify that a discrete piece of code has the correct behaviour, model validation tests verify that the output of an entire dynamic model corresponds to known behaviour from the academic literature. For instance, alongside the ion channel curation and parameter extraction tasks in ChannelWorm, a parallel effort is aimed at implementing validation tests for each of these models using the Python library SciUnit [ 48 ]. The validation process uses curated datasets of ion channel behaviour to instantiate analogous statistical tests that a researcher would use when developing such a model.…”

“…With regard to more traditional academic metrics, the special issue in which this article appears will include the publication of several new articles featuring foundational modelling, simulation, data management and data presentation technologies developed as a result of OpenWorm-led collaborations [ 19 , 20 , 23 , 48 ]. Before this special issue, we have published a handful of papers on several different facets of the project in a spectrum of journals focused on the computational biological sciences.…”

Connectome to behaviour: modelling Caenorhabditis elegans at cellular resolution

“…Models need to be constrained at many levels by experimental data to be of use in making predictions about the behaviour of the biological entities being simulated. Gerkin et al [29] describe a framework, SciUnit, which is being used in OpenWorm to validate models produced by the project against experimental data on ion channel kinetics, neuronal firing and worm locomotion.…”