Collective computational properties of neural networks: New learning mechanisms

Personnaz, L.; Guyon, Isabelle; Dreyfus, Gérard

doi:10.1103/physreva.34.4217

Cited by 309 publications

(140 citation statements)

References 11 publications

Supporting

Mentioning

137

Contrasting

Unclassified

Order By: Relevance

“…The algorithms employed in this section are all designed to generate weight matrices that are good approximations of the weight matrix generated by the pseudo-inverse rule of Personnaz et al [30]. According to this rule…”

Section: The Pseudo-inverse Class Of Modelsmentioning

confidence: 99%

“…For very small networks it is possible to explore the state space exhaustively (see, for example, [30]), in order to calculate R exactly, but for more realistic network sizes the nature of the attractors is very hard to compute [14,25] and only empirical methods, as described here, are available.…”

Section: Attractor Basin Sizementioning

confidence: 99%

“…Networks of the Hopfield type have proved in the past to be amenable to rigorous analysis, and much is known about the behaviour of the standard model under both deterministic and stochastic dynamics [2,4,24,30]. In large part this is due to the strong similarity of such systems to spin glasses, which has enabled some powerful tools of statistical physics to be applied.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

High capacity recurrent associative memories

2004

View full text Add to dashboard Cite

Various algorithms for constructing weight matrices for Hopfield-type associative memories are reviewed, including ones with much higher capacity than the basic model. These alternative algorithms either iteratively approximate the projection weight matrix or use simple perceptron learning.An experimental investigation of the performance of networks trained by these algorithms is presented, including measurements of capacity, training time and their ability to correct corrupted versions of the training patterns.

show abstract

Section: The Pseudo-inverse Class Of Modelsmentioning

confidence: 99%

Section: Attractor Basin Sizementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

High capacity recurrent associative memories

2004

View full text Add to dashboard Cite

show abstract

“…On embedding correlated patterns by this scheme, the storage performance of the network is significantly decreased. Thus, many researches for storing correlated patterns direct to orthogonalization of these patterns, such as pseudo-inverse matrix method [2] and iterative learning scheme [3]. Though these methods enable all the correlated patterns to be stable local minima in the network, their computational costs grow with respect to the network size and the number of patterns to be embedded.…”

Section: Introductionmentioning

confidence: 99%

Pseudo-Orthogonalization of Memory Patterns for Complex-Valued and Quaternionic Associative Memories

Minemoto

Isokawa

Nishimura

et al. 2017

Journal of Artificial Intelligence and Soft Computing Research

View full text Add to dashboard Cite

Hebbian learning rule is well known as a memory storing scheme for associative memory models. This scheme is simple and fast, however, its performance gets decreased when memory patterns are not orthogonal each other. Pseudo-orthogonalization is a decorrelating method for memory patterns which uses XNOR masking between the memory patterns and randomly generated patterns. By a combination of this method and Hebbian learning rule, storage capacity of associative memory concerning non-orthogonal patterns is improved without high computational cost. The memory patterns can also be retrieved based on a simulated annealing method by using an external stimulus pattern. By utilizing complex numbers and quaternions, we can extend the pseudo-orthogonalization for complex-valued and quaternionic Hopfield neural networks. In this paper, the extended pseudo-orthogonalization methods for associative memories based on complex numbers and quaternions are examined from the viewpoint of correlations in memory patterns. We show that the method has stable recall performance on highly correlated memory patterns compared to the conventional real-valued method.

show abstract

“…It was developed by Personnaz et al, [3] [4] and later refined by Kanter and Sompolinsky [5], and Gorodnichy, [6]. This method allows up to p < N linearly independent patterns to be memorized.…”

Section: Introductionmentioning

confidence: 99%