Information storage and retrieval in spin-glass like neural networks

Abstract

2014 Dans la perspective de la réalisation de mémoires associatives à l'aide de réseaux de neurones, nous étudions la relation entre la structure d'un réseau et ses états attracteurs; nous montrons que, quel que soit l'ensemble des états que l'on désire mémoriser, il est généralement possible de calculer tous les paramètres du réseau de façon à assurer la stabilité de ces états. Le formalisme des verres de spins conduit à des résultats particulièrement simples qui permettent, dans certains cas, d'evaluer analytiquement leur attractivité. Abstract. 2014 The link between the structure of a neural network and its attractor states is investigated, with a view to designing associative memories based on such networks. It is shown that, for any preassigned set of states to be memorized, the parameters of the network can be completely calculated in most cases so as to guaranteee the stability of these states. The spin glass formulation of the neural network problem leads to particularly simple results which, in some cases, allow an analytical evaluation of the attractivity of the memorized states.

Shaping attraction basins in neural networks

“…. , u {"~ are the eigenvectors of W ~ and A is the spectrum of W ~ (Venkatesh & Psaltis, 1985;Personnaz, Guyon, & Dreyfus, 1985;Venkatesh & Psaltis, 1989 2 (") = 2 > 0, we see that the matrix W" is symmetric with nonnegative eigenvalues (i.e., it is nonnegative definite). Therefore there exist Lya, punov functions in this case, and moreover it has been shown that the stored memories form global energy minima (Venkatesh & Psattis, 1989).…”

“…The memory storage capacity for this method is n/4 log n (McEliece, Posner, Rodermich, & Venkatesh, 1987;Psaltis & Venkatesh, 1989) whereas the maximal theoretical capacity for any storage algorithm is 2n (Cover, 1965;Venkatesh, 1986b). The spectral algorithm (Kohonen, 1977;Personnaz, Guyon, & Dreyfus, 1985;Venkatesh & Psaltis, 1989) and an algorithm we will refer to as the dual spectral algorithm (Maruani, Chevallier, & Sirat, 19871 are algorithms whose capacities Acknowledgement: The work of the first two authors was supported in part by NSF grant . The work at Caltech is supported by DARPA and AFOSR.…”

Strong attractors of Hopfield neural networks to model attachment types and behavioural patterns

“…The Hebbian learning paradigm in Hopfield networks provides a biologically somewhat plausible and a mathematically simple rule amongst other choices (see for example [5], [22]). We thus consider a Hopfield network with a training set of random patterns except that some of these patterns are multiply stored while others, called simple patterns, are stored once as usual.…”

Phase diagrams for locally Hopfield neural networks in presence of correlated patterns

“…However, this problem gets very simple in a special case, that is if we impose zero temperature (β = 0). In such a case we just have to solve the following system of equations (cf [12], [9] for a different approach at zero temperature)…”