Using the generating functional analysis an exact recursion relation is derived for the time evolution of the effective local field of the fully connected Little-Hopfield model. It is shown that, by leaving out the feedback correlations arising from earlier times in this effective dynamics, one precisely finds the recursion relations usually employed in the signal-to-noise approach. The consequences of this approximation as well as the physics behind it are discussed. In particular, it is pointed out why it is hard to notice the effects, especially for model parameters corresponding to retrieval. Numerical simulations confirm these findings. The signal-to-noise analysis is then extended to include all correlations, making it a full theory for dynamics at the level of the generating functional analysis. The results are applied to the frequently employed extremely diluted (a)symmetric architectures and to sequence processing networks.

Hamiltonians for general multi-state spin-glass systems with Ising symmetry are derived for both sequential and synchronous updating of the spins. The possibly different behaviour caused by the way of updating is studied in detail for the (anti)-ferromagnetic version of the models, which can be solved analytically without any approximation, both thermodynamically via a free-energy calculation and dynamically using the generating functional approach. Phase diagrams are discussed and the appearance of two-cycles in the case of synchronous updating is examined. A comparative study is made for the Q-Ising and the Blume-Emery-Griffiths ferromagnets and some interesting physical differences are found. Numerical simulations confirm the results obtained.

The effects of a variable amount of random dilution of the synaptic couplings in Q-Ising multi-state neural networks with Hebbian learning are examined. A fraction of the couplings is explicitly allowed to be anti-Hebbian. Random dilution represents the dying or pruning of synapses and, hence, a static disruption of the learning process which can be considered as a form of multiplicative noise in the learning rule. Both parallel and sequential updating of the neurons can be treated. Symmetric dilution in the statics of the network is studied using the mean-field theory approach of statistical mechanics. General dilution, including asymmetric pruning of the couplings, is examined using the generating functional (path integral) approach of disordered systems. It is shown that random dilution acts as additive gaussian noise in the Hebbian learning rule with a mean zero and a variance depending on the connectivity of the network and on the symmetry. Furthermore, a scaling factor appears that essentially measures the average amount of anti-Hebbian couplings.

The thermodynamic and retrieval properties of the Blume-Emery-Griffiths neural network with synchronous updating and variable dilution are studied using replica mean-field theory. Several forms of dilution are allowed by pruning the different types of couplings present in the Hamiltonian. The appearance and properties of two-cycles are discussed. Capacity-temperature phase diagrams are derived for several values of the pattern activity. The results are compared with those for sequential updating. The effect of self-coupling is studied. Furthermore, the optimal combination of dilution parameters giving the largest critical capacity is obtained.PACS. 05.20.-y Classical statistical mechanics -64.60.Cn Order-disorder transformations; statistical mechanics of model systems -87.18.Sn Neural Networks

We calculate the number of metastable configurations of Ising small-world networks that are constructed upon superimposing sparse Poisson random graphs onto a one-dimensional chain. Our solution is based on replicated transfer-matrix techniques. We examine the denegeracy of the ground state and find a jump in the entropy of metastable configurations exactly at the crossover between the small-world and the Poisson random graph structures. We also examine the difference in entropy between metastable and all possible configurations, for both ferromagnetic and bond-disordered long-range couplings.