PyGeNN: A Python Library for GPU-Enhanced Neural Networks

Abstract

More than half of the Top 10 supercomputing sites worldwide use GPU accelerators and they are becoming ubiquitous in workstations and edge computing devices. GeNN is a C++ library for generating efficient spiking neural network simulation code for GPUs. However, until now, the full flexibility of GeNN could only be harnessed by writing model descriptions and simulation code in C++. Here we present PyGeNN, a Python package which exposes all of GeNN's functionality to Python with minimal overhead. This provides an alternative, arguably more user-friendly, way of using GeNN and allows modelers to use GeNN within the growing Python-based machine learning and computational neuroscience ecosystems. In addition, we demonstrate that, in both Python and C++ GeNN simulations, the overheads of recording spiking data can strongly affect runtimes and show how a new spike recording system can reduce these overheads by up to 10×. Using the new recording system, we demonstrate that by using PyGeNN on a modern GPU, we can simulate a full-scale model of a cortical column faster even than real-time neuromorphic systems. Finally, we show that long simulations of a smaller model with complex stimuli and a custom three-factor learning rule defined in PyGeNN can be simulated almost two orders of magnitude faster than real-time.

mlGeNN: accelerating SNN inference using GPU-enabled neural networks

“…In our previous work [38] we introduced 'procedural connectivity', a technique where neurons' outgoing sparse random connectivity is regenerated on the fly when they spike, rather than being stored in memory. Using procedural connectivity we demonstrated how a model-so large it could previously only be simulated on a supercomputer-could be simulated on a single GPU.…”

“…Because this algorithm is instantiated using GeNN's code generator, constant terms will be hard-coded into the kernel, allowing the CUDA compiler to transform divisions by constants into more efficient instructions, unroll loops and optimise for special cases such as O chan = 1. Furthermore, unlike with the probabilistic connectivity investigated in our previous work [38], combining procedural convolutions with learning is entirely possible as the algorithm to back-propagate through the convolutional connectivity is very similar and equally efficient.…”

“…These ensure that all inputs to the neurons at the junction points are correctly synchronised. [38] to accelerate the simulation of SNN models using GPUs.…”

“…It is freely available online at https://github.com/genn-team/ml_genn. Under the hood, rather than using a tensor based ML library to provide GPU acceleration, mlGeNN uses PyGeNN [38]-a Python interface to the GeNN GPU-accelerated SNN simulator [39] which has previously demonstrated excellent performance at accelerating a range of computational neuroscience models [40,41]. We have extended GeNN with two methods for efficiently implementing convolutional connectivity.…”

Sub-realtime simulation of a neuronal network of natural density

“…Fast and energy efficient simulation is a promise of neuromorphic computing [5]; desirable for large-scale neuroscientific models [6] and imperative in artificial intelligence and machine learning applications [7]. The first milestone is realtime performance, which was accomplished for the microcircuit model in 2019 on a neuromorphic system [8] followed this year by GPU systems [9,10], one of them already breaking into the sub-realtime regime [10]. However, these results have to be evaluated in the light of continuously advancing commodity hardware as a reference technology providing more flexibility at potentially lower costs.…”