Topological and Dynamical Complexity of Random Neural Networks

Abstract

Random neural networks are dynamical descriptions of randomly interconnected neural units. These show a phase transition to chaos as a disorder parameter is increased. The microscopic mechanisms underlying this phase transition are unknown, and similarly to spin-glasses, shall be fundamentally related to the behavior of the system. In this Letter we investigate the explosion of complexity arising near that phase transition. We show that the mean number of equilibria undergoes a sharp transition from one equilibrium to a very large number scaling exponentially with the dimension on the system. Near criticality, we compute the exponential rate of divergence, called topological complexity. Strikingly, we show that it behaves exactly as the maximal Lyapunov exponent, a classical measure of dynamical complexity. This relationship unravels a microscopic mechanism leading to chaos which we further demonstrate on a simpler disordered systems, suggesting a deep and underexplored link between topological and dynamical complexity. Heterogeneity of interconnections is a crucial property to understand the behavior of realistic networks that arise in the modeling of physical, biological or social complex systems. Among these, a paramount example is given by neuronal networks of the brain. In these systems, synaptic connections display characteristic disorder properties [1,2] resulting from development and learning. Taking into account this heterogeneity seems now essential, as experimental studies of neuronal tissues have shown that the degree of disorder in the connections significantly impacts the input-output function, rhythmicity and synchrony, effects that can be related to transitions between physiological and pathological behaviors [3][4][5]. As an example, Aradi and Soltesz [3] have shown that rats subject to febrile seizure present the same average synaptic properties but increased variance.These properties are reminiscent of disordered physical systems such as spin-glasses. The relationship between disorder and qualitative behaviors has been thoroughly studied within theoretical frameworks such as the Sherrington-Kirkpatrick model [6] describing the behavior of binary variables interacting through a random connectivity matrix. In these models, estimating the number of metastable states, deemed to be deeply related to the transition to chaos, remains an important endeavor [7]. In the context of nervous system modeling, random neural networks [8,9] constitute a prominent class of models, which describe the evolution of the activity of a neuron i in a n-neurons network through the randomly coupled system of ordinary differential equations:where J ij are independent centered Gaussian random variables of variance σ 2 /n representing the synaptic connectivity coefficients between neuron i and j [10], and S is an odd sigmoid function with maximal unit slope at the origin (S (x) ≤ S (0) = 1) representing the synaptic nonlinearity.The behavior of system (1) has been analyzed in the asymptotic regime of infinite population si...

Eigenspectrum bounds for semirandom matrices with modular and spatial structure for neural networks

“…The eigenspectrum of the weighted coupling matrix W of a neural network provides valuable information concerning the stability and behavior of a simulated model [1][2][3][4][5][6]: for example, the magnitude of the eigenvalue with the largest real part places limits on the linear stability of a network or of network partitions [1,3,5,7]; the existence and magnitude of complex eigenvalues determine whether oscillatory network dynamics are expressed [4,[8][9][10]. Numerically computing eigenvalues for small networks is trivial, but analytical eigenvalue solutions for matrices corresponding to even small nonsymmetric networks with relatively simple structure rapidly become intractable [10].…”

Transition from Gaussian to non-Gaussian fluctuations for mean-field diffusions in spatial interaction

“…The motivation of going beyond pure mean-field interactions comes from the biological observation that neurons do not interact in a mean-field way (see [11,41] and references therein). There has been recently a growing interest in models closer to the topology of real neuronal networks [21,35,41].…”

“…The motivation of going beyond pure mean-field interactions comes from the biological observation that neurons do not interact in a mean-field way (see [11,41] and references therein). There has been recently a growing interest in models closer to the topology of real neuronal networks [21,35,41]. Even though the analysis of such models seems to be difficult in general, it is quite natural to expect that properties valid in the pure mean-field case (the first of them being the existence of a continuous limit in an infinite population) still hold for perturbations of the mean-field case, namely for systems where the interactions are not strictly identical, but where the number of connections is sufficiently large to ensure some self-averaging as the population size increases.…”

Is chaos making a difference? Synchronization transitions in chaotic and nonchaotic neuronal networks

“…More specifically, our networks always showed chaotic 291 behavior at some g values regardless of the dynamics of the isolated nodes. This is not 292 surprising, as chaotic behavior arises in networks of very simple units and seems to 293 depend more strongly on other factors such as synaptic weights and network 294 topology [25][26][27]52]. Moreover, just the high-dimensionality of the systems seems to be 295 enough to assure that chaos will emerge under some conditions, for example, the 296 quasiperiodic route to chaos in high-dimensional systems [53].…”