On the computational power and complexity of Spiking Neural Networks

Abstract

The last decade has seen the rise of neuromorphic architectures based on artificial spiking neural networks, such as the SpiNNaker, TrueNorth, and Loihi systems. The massive parallelism and colocating of computation and memory in these architectures potentially allows for an energy usage that is orders of magnitude lower compared to traditional Von Neumann architectures. However, to date a comparison with more traditional computational architectures (particularly with respect to energy usage) is hampered by the lack of a formal machine model and a computational complexity theory for neuromorphic computation. In this paper we take the first steps towards such a theory. We introduce spiking neural networks as a machine model where-in contrast to the familiar Turing machine-information and the manipulation thereof are co-located in the machine. We introduce canonical problems, define hierarchies of complexity classes and provide some first completeness results.

Investigating Multisensory Integration in Emotion Recognition Through Bio-Inspired Computational Models

“…Then more complex permutations of interaction will need to be explored. The high complexity has been a concern for using SNN in real-time applications [10], [86], and we will explore solutions to improve the computational efficiency.…”

Vector Symbolic Architectures as a Computing Framework for Nanoscale Hardware

“…VSA can be formulated with different types of vectors, namely those containing real, complex, or binary entries, as well as with the multivectors of geometric algebra. These flavors of VSA come under many different names: Holographic Reduced Representation (HRR) [Plate, 1995a], [Plate, 2003], Multiply-Add-Permute (MAP) [Gayler, 1998], Binary Spatter Codes [Kanerva, 1997], Sparse Binary Distributed Representations (SBDR) [Rachkovskij and Kussul, 2001], [Rachkovskij, 2001], Sparse Block-Codes [Laiho et al, 2015], [Frady et al, 2020b], Matrix Binding of Additive Terms (MBAT) [Gallant and Okaywe, 2013], Geometric Analogue of Holographic Reduced Representation (GAHRR) [Aerts et al, 2009], etc. All of these different models have similar computational properties -see [Frady et al, 2018b] and [Schlegel et al, 2020].…”

“…Here the similarity of sequences is defined by the number of the same elements in the same sequential positions, where the sequences are aligned by their last elements. Evidently, this definition does not take into account the same elements in different positions, in contrast to, e.g., an edit distance of sequences [Levenshtein, 1966]. Note that the edit distance can be approximated by vectors of n-gram frequencies and their randomized versions akin to hypervectors (see, e.g., [Sokolov, 2007], [Hannagan et al, 2011]).…”

“…Thus, VSA models that use sparse representations are extremely important for mapping VSA operations efficiently onto hardware. We are aware of two such models: Sparse Binary Distributed Representations [Rachkovskij and Kussul, 2001], [Rachkovskij, 2001] and Sparse Block-Codes (SBC) [Laiho et al, 2015], [Frady et al, 2020b]. In Sparse Binary Distributed Representations model, the hypervectors are sparse patterns without any restrictions on placing the active components while in Sparse Block-Codes the hypervectors are divided into blocks of the same size (denoted as K) with just one single active component per block.…”

Neuromorphic Computing is Turing-Complete

“…Neuromorphic computers, due to their low energy consumption, are prime candidates for co-processors and accelerators in extremely heterogeneous computing systems [42,25]. However, because of the absence of a sound computability and complexity theory for neuromorphic computing [43], we are faced with a murky understanding of the big-O runtimes [44] of neuromorphic algorithms and unable to compare them to their conventional counterparts. This is not the case for other post-Moore computing paradigms such as quantum computing [45,46,47], which has a well-founded literature on the theory of quantum computation [48,49].…”