A Collaborative Simulation-Analysis Workflow for Computational Neuroscience Using HPC

Senk, Johanna; Yegenoglu, Alper; Amblet, Olivier; Brukau, Yury; Davison, Andrew P.; Lester, David; Lührs, Anna; Quaglio, Pietro; Rostami, Vahid; Rowley, Andrew; Schuller, Bernd; Stokes, Alan B.; Albada, Sacha van; Zielasko, Daniel; Diesmann, Markus; Weyers, Benjamin; Denker, Michael; Grün, Sonja

doi:10.1007/978-3-319-53862-4_21

Cited by 14 publications

(20 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Collaboratory . The collaboratory is a web portal designed within the Human Brain Project intended to improve the quality of collaboration between many possibly international parties (Senk et al, 2017 ). It allows scientists to share data, collaborate on code and re-use models and methods, and enables tracking and crediting researchers for their contributions.…”

Section: Resultsmentioning

confidence: 99%

Reproducing Polychronization: A Guide to Maximizing the Reproducibility of Spiking Network Models

Pauli

Weidel

Kunkel

et al. 2018

Front. Neuroinform.

View full text Add to dashboard Cite

Any modeler who has attempted to reproduce a spiking neural network model from its description in a paper has discovered what a painful endeavor this is. Even when all parameters appear to have been specified, which is rare, typically the initial attempt to reproduce the network does not yield results that are recognizably akin to those in the original publication. Causes include inaccurately reported or hidden parameters (e.g., wrong unit or the existence of an initialization distribution), differences in implementation of model dynamics, and ambiguities in the text description of the network experiment. The very fact that adequate reproduction often cannot be achieved until a series of such causes have been tracked down and resolved is in itself disconcerting, as it reveals unreported model dependencies on specific implementation choices that either were not clear to the original authors, or that they chose not to disclose. In either case, such dependencies diminish the credibility of the model's claims about the behavior of the target system. To demonstrate these issues, we provide a worked example of reproducing a seminal study for which, unusually, source code was provided at time of publication. Despite this seemingly optimal starting position, reproducing the results was time consuming and frustrating. Further examination of the correctly reproduced model reveals that it is highly sensitive to implementation choices such as the realization of background noise, the integration timestep, and the thresholding parameter of the analysis algorithm. From this process, we derive a guideline of best practices that would substantially reduce the investment in reproducing neural network studies, whilst simultaneously increasing their scientific quality. We propose that this guideline can be used by authors and reviewers to assess and improve the reproducibility of future network models.

show abstract

Section: Resultsmentioning

confidence: 99%

Reproducing Polychronization: A Guide to Maximizing the Reproducibility of Spiking Network Models

Pauli

Weidel

Kunkel

et al. 2018

Front. Neuroinform.

View full text Add to dashboard Cite

show abstract

“…This renders the total memory consumption approximately directly proportional to the number of neuronal and synaptic elements in the model. The model already serves as a building block for a number of further studies and larger networks (Wagatsuma et al, 2011 ; Cain et al, 2016 ; Hagen et al, 2016 ; Schmidt et al, 2016 ; Schwalger et al, 2017 ), and a first comparison of the simulation results of NEST and SpiNNaker for this model has served as a test case for a workflow implementation on the collaboration platform of the Human Brain Project (Senk et al, 2017 ). The original implementation uses NEST, which can also handle much larger networks with trillions of synapses (RIKEN BSI, 2013 ; Kunkel et al, 2014 ; Forschungszentrum Jülich, 2018 ; Jordan et al, 2018 ) under the increased memory consumption and run time costs indicated above.…”

Section: Introductionmentioning

confidence: 99%

Performance Comparison of the Digital Neuromorphic Hardware SpiNNaker and the Neural Network Simulation Software NEST for a Full-Scale Cortical Microcircuit Model

et al. 2018

Self Cite

View full text Add to dashboard Cite

The digital neuromorphic hardware SpiNNaker has been developed with the aim of enabling large-scale neural network simulations in real time and with low power consumption. Real-time performance is achieved with 1 ms integration time steps, and thus applies to neural networks for which faster time scales of the dynamics can be neglected. By slowing down the simulation, shorter integration time steps and hence faster time scales, which are often biologically relevant, can be incorporated. We here describe the first full-scale simulations of a cortical microcircuit with biological time scales on SpiNNaker. Since about half the synapses onto the neurons arise within the microcircuit, larger cortical circuits have only moderately more synapses per neuron. Therefore, the full-scale microcircuit paves the way for simulating cortical circuits of arbitrary size. With approximately 80, 000 neurons and 0.3 billion synapses, this model is the largest simulated on SpiNNaker to date. The scale-up is enabled by recent developments in the SpiNNaker software stack that allow simulations to be spread across multiple boards. Comparison with simulations using the NEST software on a high-performance cluster shows that both simulators can reach a similar accuracy, despite the fixed-point arithmetic of SpiNNaker, demonstrating the usability of SpiNNaker for computational neuroscience applications with biological time scales and large network size. The runtime and power consumption are also assessed for both simulators on the example of the cortical microcircuit model. To obtain an accuracy similar to that of NEST with 0.1 ms time steps, SpiNNaker requires a slowdown factor of around 20 compared to real time. The runtime for NEST saturates around 3 times real time using hybrid parallelization with MPI and multi-threading. However, achieving this runtime comes at the cost of increased power and energy consumption. The lowest total energy consumption for NEST is reached at around 144 parallel threads and 4.6 times slowdown. At this setting, NEST and SpiNNaker have a comparable energy consumption per synaptic event. Our results widen the application domain of SpiNNaker and help guide its development, showing that further optimizations such as synapse-centric network representation are necessary to enable real-time simulation of large biological neural networks.

show abstract

“…Generic attempts to overcome these difficulties and formalize the validation process include the development of the Python module (Omar et al, 2014; Sarma et al, 2016), and the description of workflows for the validation of models (Senk et al, 2017; Kriegeskorte and Douglas, 2018; van Albada et al, 2018). In this study, we build on these efforts in order to introduce a workflow and supporting software to quantitatively compare and validate spiking network models.…”

Section: Introductionmentioning

confidence: 99%

Reproducible Neural Network Simulations: Statistical Methods for Model Validation on the Level of Network Activity Data

Gutzen

Papen

Trensch

et al. 2018

Front. Neuroinform.

Self Cite

View full text Add to dashboard Cite

Computational neuroscience relies on simulations of neural network models to bridge the gap between the theory of neural networks and the experimentally observed activity dynamics in the brain. The rigorous validation of simulation results against reference data is thus an indispensable part of any simulation workflow. Moreover, the availability of different simulation environments and levels of model description require also validation of model implementations against each other to evaluate their equivalence. Despite rapid advances in the formalized description of models, data, and analysis workflows, there is no accepted consensus regarding the terminology and practical implementation of validation workflows in the context of neural simulations. This situation prevents the generic, unbiased comparison between published models, which is a key element of enhancing reproducibility of computational research in neuroscience. In this study, we argue for the establishment of standardized statistical test metrics that enable the quantitative validation of network models on the level of the population dynamics. Despite the importance of validating the elementary components of a simulation, such as single cell dynamics, building networks from validated building blocks does not entail the validity of the simulation on the network scale. Therefore, we introduce a corresponding set of validation tests and present an example workflow that practically demonstrates the iterative model validation of a spiking neural network model against its reproduction on the SpiNNaker neuromorphic hardware system. We formally implement the workflow using a generic Python library that we introduce for validation tests on neural network activity data. Together with the companion study (Trensch et al., 2018), the work presents a consistent definition, formalization, and implementation of the verification and validation process for neural network simulations.

show abstract

A Collaborative Simulation-Analysis Workflow for Computational Neuroscience Using HPC

Cited by 14 publications

References 12 publications

Reproducing Polychronization: A Guide to Maximizing the Reproducibility of Spiking Network Models

Reproducing Polychronization: A Guide to Maximizing the Reproducibility of Spiking Network Models

Performance Comparison of the Digital Neuromorphic Hardware SpiNNaker and the Neural Network Simulation Software NEST for a Full-Scale Cortical Microcircuit Model

Reproducible Neural Network Simulations: Statistical Methods for Model Validation on the Level of Network Activity Data

Contact Info

Product

Resources

About