Spontaneous scale-free structure of spike flow graphs in recurrent neural networks

Spectra of winner-take-all stochastic neural networks

“…In Piekniewski & Schreiber (2008) we have developed a simple and tractable mathematical model shedding some light on a particular class of the afore-mentioned phenomena, namely on mesoscopic level self-organisation of functional brain networks under fMRI imaging, where we have achieved a high degree of agreement with existing empirical reports. Being addressed to the neuroscientific community, our work Piekniewski & Schreiber (2008) relied on semirigorous study of information flow structure in a class of recurrent neural networks exhibiting asymptotic scale-free behaviour and admitting a description in terms of the so-called winner-take-all dynamics. The purpose of the present paper is to define and study these winner-take-all networks with full mathematical rigour in context of their asymptotic spectral properties, well known to be of interest for neuroscientific community.…”

“…Certain heuristical non-rigorous considerations aimed at explaining these phenomena have been offered in Fraiman (2009) and Kitzblicher (2009) discussing very interesting analogies between crucial features of functional brain networks and Ising model at criticality. Up to our best knowledge, the first dedicated mathematical model shedding some light on the scale-free properties of mesoscopic brain functional networks is the simple spin glass type system introduced in Piekniewski & Schreiber (2008) further extended and enhanced with a geometric ingredient in Piersa, Piekniewski & Schreiber (2010) and standing in good agreement with empirical findings. The details and neuroscientific motivations of these models are far beyond the scope of the present mathematically oriented paper and we only provide a brief overview for completeness here, proceeding to well-defined rigorous problems as soon as possible.…”

“…

During the recent few years, in response to empirical findings suggesting scalefree self-organisation phenomena emerging in complex nervous systems at a mesoscale level, there has been significant search for suitable models and theoretical explanations in neuroscientific literature, see the recent survey by Bullmore & Sporns (2009). In Piekniewski & Schreiber (2008) we have developed a simple and tractable mathematical model shedding some light on a particular class of the afore-mentioned phenomena, namely on mesoscopic level self-organisation of functional brain networks under fMRI imaging, where we have achieved a high degree of agreement with existing empirical reports. Being addressed to the neuroscientific community, our work Piekniewski & Schreiber (2008) relied on semirigorous study of information flow structure in a class of recurrent neural networks exhibiting asymptotic scale-free behaviour and admitting a description in terms of the so-called winner-take-all dynamics.

…”

Capacity of an associative memory model on random graph architectures

“…It has been realized that this "power law"-behavior is prevalent in realistic graphs arising in various areas. Graphs with power law degree distribution are ubiquitously encountered, for example, in the internet, the telecommunications graphs, the neural networks and many biological applications [20,34,39]. The common feature of such networks is that they are large, have small diameter, but have small average degree.…”

Structural Properties of Recurrent Neural Networks

“…In [7] an interesting model of asynchronous RNN with a winner-take-all type dynamics was introduced. This model was found to have strong scale-free properties.…”