Correlated Fluctuations in Strongly Coupled Binary Networks Beyond Equilibrium

Abstract

Randomly coupled Ising spins constitute the classical model of collective phenomena in disordered systems, with applications covering glassy magnetism and frustration, combinatorial optimization, protein folding, stock market dynamics, and social dynamics. The phase diagram of these systems is obtained in the thermodynamic limit by averaging over the quenched randomness of the couplings. However, many applications require the statistics of activity for a single realization of the possibly asymmetric couplings in finite-sized networks. Examples include reconstruction of couplings from the observed dynamics, representation of probability distributions for sampling-based inference, and learning in the central nervous system based on the dynamic and correlation-dependent modification of synaptic connections. The systematic cumulant expansion for kinetic binary (Ising) threshold units with strong, random, and asymmetric couplings presented here goes beyond mean-field theory and is applicable outside thermodynamic equilibrium; a system of approximate nonlinear equations predicts average activities and pairwise covariances in quantitative agreement with full simulations down to hundreds of units. The linearized theory yields an expansion of the correlation and response functions in collective eigenmodes, leads to an efficient algorithm solving the inverse problem, and shows that correlations are invariant under scaling of the interaction strengths.

Self-consistent formulations for stochastic nonlinear neuronal dynamics

“…This is, moreover, identical to (1) for the case of a fully connected network J ij = N −1 J and perfectly correlated stochastic increments dW i across neurons, if one defines f (x) = −x + J φ (x). For the function φ we assume an expansive non-linearity of convex down shape.…”

“…The aim of this article is to survey methods to construct self-consistent solutions to such stochastic non-linear differential equations. The N components in (1) do not qualitatively increase the difficulty -rather the interplay of fluctuations and the non-linearity is the cause of complications. To illustrate the concepts in the simplest but non-trivial setting, in the remainder of this article we therefore concentrate on the scalar equation…”

“…We determine the true mean valuesx andx of the two fields by solving the two implicit equations Γ (1) (18)). All further Taylor coefficients (or Volterra kernels) in (20) have physical meanings.…”

“…The simplest measure to extract from Γ λ is the mean valuex λ defined by the equation of stateΓ (1) x,λ [x * λ ,x * λ ] = 0 6 . Differentiating this equation with respect to λ leads to…”

Integration of Continuous-Time Dynamics in a Spiking Neural Network Simulator

“…Networks of such noisy linear rate models have been investigated to explain features such as oscillations (Bos et al, 2016) or the smallness of average correlations (Tetzlaff et al, 2012; Helias et al, 2013). We here consider a prototypical network model of excitatory and inhibitory units following the linear dynamics given by Equation 9 with ϕ( x ) = ψ( x ) = x , μ = 0, and noise amplitude σ, $\tau d{X}^i(t)=\left(-X^i+{\displaystyle \sum _{j=1}^N{w}^{ij}}{X}^j(t)\right)dt+\sqrt{\tau }\sigma d{W}^i(t).$ Due to the linearity of the model, the cross-covariance between units i and j can be calculated analytically and is given by Ginzburg and Sompolinsky (1994); Risken (1996); Gardiner (2004); Dahmen et al (2016): $c(t)={\displaystyle \sum _{i,j}\frac{v^{i\text{T}}{\sigma }^2{v}^j}{{\lambda }_i+{\lambda }_j}}{u}^i{u}^{j\text{T}}\left(H(t)\frac{1}{\tau }{e}^{-{\lambda }_i\frac{t}{\tau }}+H(-t)\frac{1}{\tau }{e}^{{\lambda }_j\frac{t}{\tau }}\right),$ where H denotes the Heaviside function. The λ i indicate the eigenvalues of the matrix 𝟙 − W corresponding to the i -th left and right eigenvectors v i and u i respectively.…”