A Network Simulator for the Estimation of Bandwidth Load and Latency Created by Heterogeneous Spiking Neural Networks on Neuromorphic Computing Communication Networks

Abstract

Accelerated simulations of biological neural networks are in demand to discover the principals of biological learning. Novel many-core simulation platforms, e.g., SpiNNaker, BrainScaleS and Neurogrid, allow one to study neuron behavior in the brain at an accelerated rate, with a high level of detail. However, they do not come anywhere near simulating the human brain. The massive amount of spike communication has turned out to be a bottleneck. We specifically developed a network simulator to analyze in high detail the network loads and latencies caused by different network topologies and communication protocols in neuromorphic computing communication networks. This simulator allows simulating the impacts of heterogeneous neural networks and evaluating neuron mapping algorithms, which is a unique feature among state-of-the-art network models and simulators. The simulator was cross-checked by comparing the results of a homogeneous neural network-based run with corresponding bandwidth load results from comparable works. Additionally, the increased level of detail achieved by the new simulator is presented. Then, we show the impact heterogeneous connectivity can have on the network load, first for a small-scale test case, and later for a large-scale test case, and how different neuron mapping algorithms can influence this effect. Finally, we look at the latency estimations performed by the simulator for different mapping algorithms, and the impact of the node size.

Verification of a neuromorphic computing network simulator using experimental traffic data

“…In a previous study, we introduced an in-house implemented network simulator specifically designed to evaluate network load and latency on an NC communication network. In this subsection, we will briefly describe this simulator, a more detailed description can be found in Kleijnen et al ( 2022 ) and its supplementary material. The simulator operates in four steps: generation of a neural network (NN), generation of the hardware graph, assignment of neurons to computational nodes, i.e., neuron mapping, and the simulation of the spike packet movement created by each neuron.…”

“…In Kleijnen et al ( 2022 ), the performance of our simulator was validated against a numerical model (Vainbrand and Ginosar, 2011 ) and put into relation to an analytical model presented in Kauth et al ( 2020 ). However, both these models are limited to the use of homogeneous connectivity models.…”

“…In order to better understand the performance of communication networks and communication protocols in these systems in an early design phase, we developed a Python-based network simulator introduced in a previous work (Kleijnen et al, 2022 ). With the help of the simulator, the communication network load can be estimated at a high level of abstraction during the conception phase of the design.…”

neuroAIx-Framework: design of future neuroscience simulation systems exhibiting execution of the cortical microcircuit model 20× faster than biological real-time

“…In our previous work, we studied suitable communication architectures—a major bottleneck in accelerated simulation of large-scale networks—utilizing the static and dynamic simulators (Kauth et al, 2020 ). Similarly, Kleijnen et al ( 2022 ) focused on simulation regarding heterogeneous neural networks and corresponding mapping algorithms. However, this work extends this to the characterization of all relevant building blocks that are necessary for a dedicated neuroscience simulation system, and sketches their implementation in the presented evaluation platform.…”

Data_Sheet_1.PDF

“…This work may thus be considered a supplement to some recent achievements reported earlier. In Kleijnen et al (2022), we presented a Python-based network simulator for large-scale heterogeneous neural networks and applied it to the evaluation of network traffic caused by the multiarea model (Schmidt et al, 2018). This model is composed of 32 areas, which-loosely speaking-may be considered special instances of the cortical microcircuit with some extra connectivity.…”