It is shown that an Ising spin system in which both spins and interactions evolve in time according to dynamical laws suggested by neural processes, but with widely separated timescales, leads to a thermodynamic equilibrium corresponding to a system of averaged replicas, where the replica number can take any real value determined by the ratio of characteristic temperatures. The resultant phase structure has interesting features which are explored.In this paper we show that a simple model of coupled dynamics of fast spins and slow interactions, stimulated by considerations of simultaneous learning and retrieval in recurrent neural networks, leads naturally to an effective statistical mechanics characterized by a partition function which is an average over a replicated system. This latter is reminiscent of the replica trick used to consider disordered systems, such as spin glasses, but with the important difference that the number of replicas has a physica/ meaning as the ratio of two characteristic temperatures and can be varied throughout the whole range of real values. We further demonstrate that the model has interesting phase consequences as a function of varying this ratio, and external stimuli, and that it can be extended to a range of other models.As the basic archetypal model we consider a system ofinteracting via continuous-valued symmetric exchange interactions J;, which themselves evolve in response to the states of the spins. The spins are taken to have a stochastic field-alignment dynamics which is fast compared with the evolution rate of the interactions J;, such that on the timescale of J, . dynamics, the spins are effectively in equilibrium according to a Boltzmann distribution, P (~) ( [ tr, j ) cc exp [ I3H t J ) ( [ cr; j -) ], where H(~) ( [tT; j ) = -g J; cr; cr. i&j and the subscript [ J;. j indicates that the [ J; j are to be considered as quenched variables. In practice, several specific types of dynamics which obey detailed balance lead to the equilibrium distribution (1), such as a Markov process with single-spin-Hip Glauber dynamics. ' The quantity P is an inverse temperature characterizing the stochastic gain. For the J; dynamics we choose the form, d 1 1 J, , = (tT, oj)(~) --PJ,j+ rj;J(t) (t (J), (3) where ( . . )(J ) refers to a thermodynamic average tj over the distribution (1) with the effectively instantaneous [J, j, and ri,j(t) is a stochastic Gaussian white noise of zero mean and correlation lij(t)nkl(t )~2+~5 (ij) (kt)~(tThe first term on the right-hand side of (3) is inspired by the Hebbian process in neural tissue in which synaptic efficacies are believed to grow locally in response to the simultaneous activity of presynaptic and postsynaptic neurons. The second term acts to limit the magnitude of Jj; P is the characteristic inverse temperature of the interaction system.(A related interaction dynamics without the noise term, equivalent to P=~, was introduced by Shinomoto. ) Generalizing spin systems by considering the interactions to be slowly time dependent was also proposed ...
