Zero temperature parallel dynamics for infinite range spin glasses and neural networks

Abstract

We present the results of analytical and numerical calculations for the zero temperature parallel dynamics of spin glass and neural network models. We use an analytical approach to calculate the magnetization and the overlaps after a few time steps. For the long time behaviour, the analytical approach becomes too complicated and we use numerical simulations. For the Sherrington-Kirkpatrick model, we measure the remanent magnetization and the overlaps at different times and we observe power law decays towards the infinite time limit. When one iterates two configurations in parallel, their distance d(∞) in the limit of infinite time depends on their initial distance d(0). Our numerical results suggest that d(∞) has a finite limit when d(0) → 0. This result can be regarded as a collective effect between an infinite number of spins. For the Little-Hopfield model, we compute the time evolution of the overlap with a stored pattern. We find regimes for which the system learns better after a few time steps than in the infinite time limit

New method for studying the dynamics of disordered spin systems without finite-size effects

“…We find a power-law decay: m(t) -m(oo) oc/ ~"^ with a =0.474 and m{oo) =0.184 for the infinite system, PACS numbers: 87. 10.+e Networks composed of spinlike elements which interact via long-range random interactions are of growing interest. Such models served as approximations for disordered materials as, for example, the Sherrington-Kirkpatrick (SK) model for spin glasses [l].…”

“…Apart from approximate treatments [8,9] exact calculations are only possible for very few time steps, e.g., up to four time steps for the SK model and two time steps for the Hopfield model [10].…”

Structure of attractors in randomly connected networks

“…Simple models of neural networks [10][11][12][13][14][15][16][17][18] have been explored to elucidate characteristics of their complex dynamics. In these networks, connections between binary neurons are independently drawn from an identical distribution, and the state of a network is updated simultaneously in discrete time steps without thermal noise.…”

Signal-to-noise analysis of Hopfield neural networks with a formulation of the dynamics in terms of transition probabilities

“…The first application of the GFA method in neural networks is reported in Ref. [3] for the Little-Hopfield model in the zero temperature case. The solution for the case of finite temperature was given in Ref.…”