Beyond the edge of chaos: Amplification and temporal integration by recurrent networks in the chaotic regime

Abstract

Randomly connected networks of neurons exhibit a transition from fixed-point to chaotic activity as the variance of their synaptic connection strengths is increased. In this study, we analytically evaluate how well a small external input can be reconstructed from a sparse linear readout of network activity. At the transition point, known as the edge of chaos, networks display a number of desirable features, including large gains and integration times. Away from this edge, in the nonchaotic regime that has been the focus of most models and studies, gains and integration times fall off dramatically, which implies that parameters must be fine tuned with considerable precision if high performance is required. Here we show that, near the edge, decoding performance is characterized by a critical exponent that takes a different value on the two sides. As a result, when the network units have an odd saturating nonlinear response function, the falloff in gains and integration times is much slower on the chaotic side of the transition. This means that, under appropriate conditions, good performance can be achieved with less fine tuning beyond the edge, within the chaotic regime.

Structure of attractors in randomly connected networks

“…We can easily see thatq = 0 is a solution of Eq. (A4) [5]. Hence, ifq = 0, the average · L0 ≡ · L |q =0 is an average over Gaussian h of mean h t L0 = 0 and covariance h t h s L0 = q ts , which simplifies the saddle-point equation of Eq.…”

“…The binomial distribution in Eq. (5) suggests that the state overlap for each neuron is approximately independent, occurring with probability (1 + ϕ(q ′ ))/2 (see Appendix A for a support). A similar expression is obtained for random Boolean networks by replacing ϕ(q) with ϕ BN (q) ≡ δ q,1 , simply reflecting completely random nature of state transitions.…”

“…In the past decades, there are a surge of research interests in randomly connected neural networks [2][3][4][5][6][7]. Although their behavior is described by simple deterministic equations, the resulting dynamics are rich, exhibiting fixed-point behavior, limit cycles, or high-dimensional chaos.…”

Is chaos making a difference? Synchronization transitions in chaotic and nonchaotic neuronal networks

“…The second type of chaotic firing pattern 17 is the synchronous chaos. Synchronous chaos has been demonstrated in networks of 18 both biophysical and non-biophysical neurons [3,13,15,17,[22][23][24], where neurons display 19 synchronous chaotic firing-rate fluctuations. The latter cases, the chaotic behavior is a 20 result of network connectivity, since isolated neurons do not display chaotic dynamics or 21 burst firing.…”

“…More recently, a totally different mechanism showed that asynchronous 22 chaos, where neurons exhibit asynchronous chaotic firing-rate fluctuations, emerge 23 generically from balanced networks with multiple time scales synaptic dynamics [20]. 24 Different modeling approaches have been used to uncover important conditions for 25 observing these types of chaotic behavior (in particular, synchronous and asynchronous 26 chaos) in neural networks, such as the synaptic strength [25][26][27] in a network, 27 heterogeneity of the numbers of synapses and their synaptic strengths [28,29], and lately 28 the balance of excitation and inhibition [21] . The results obtained by Sompolinsky et 29 al.…”