The capacity of the Hopfield associative memory

Abstract

-Techniques from,coding theory are applied to study rigorously the capacity of the Hopfield associative memory. Such a memory stores n -tuple of + 1's. The components change depending on a hardlimited version of linear functions of all other components. With symmetric connections between components, a stable state is ultimately reached. By building up the connection matrix as a sum-of-outer products of m fundamental memories, one hopes to be able to recover a certain one of the no memories by using an initial n-tuple probe vector less than a Hamming distance n/2 away from the ftindamental memory. If WI fundamental memories are chosen at random, the maximum asympotic value of m in order that most of the no original memories are exactly recoverable is n/(2log n). With the added restriction that every one of the m fundamental memories be recoverable exactly, rrl can be no more than n/(4log n) asymptotically as n approaches infinity. Extensions are also considered, in particular to capacity under qnantization of the outer-product connection matrijr. This quantized memory capacity problem is closely related to the capacity of the quantized Gaussian channel.

Bayesian retrieval in associative memories with storage errors

“…However, for these, the asymptotic capacity always goes to zero asymptotically for a error criterion for retrieval that demands vanishing errors. For instance, for the Hopfield model with linear learning, the number of patterns per neuron is , and for clipped learning, it is [19]. If a general nonlinear function is used in place of the Heaviside function for weight saturation, then, in general, the information capacity stays below that of linear learning [20].…”

“…Again, arbitrary offsets depending on the initial pattern do not change the system dynamics. Comparing (7) with (19), the MAP Lyapunov function suggests the constraint coefficients (20) (21) with the offset The resulting coefficients are positive if the initial pattern is closer to the training pattern than the inverted initial pattern, i.e., . Modifications similar to this have previously been proposed for other memory models on a heuristic basis.…”

Self-Assembled Three-Dimensional Non-Volatile Memories

“…One could use a random agglomerate of particles, all connected together, provided that the percolation is adequate (there are sufficient electrically connected paths through the network). By using proper encoding and data detection algorithms, information storage can be stored in an associative fashion [13,14].…”

Capacity bounds for the CDMA system and a neural network: a moderate deviations approach

“…So, the first and most basic question for this model is whether the patterns are fixed points of the dynamics: this question was first considered for i.i.d. symmetric Bernoulli patterns by McEliece et al [30]. We will adopt their definition of storage capacity as the maximum number of patterns M that are stable under the dynamics described above, with a probability converging to one as N → +∞.…”