On a Model of Associative Memory with Huge Storage Capacity

Demircigil, Mete; Heusel, Judith; Löwe, Matthias; Upgang, Sven; Vermet, Franck

doi:10.1007/s10955-017-1806-y

Cited by 66 publications

(58 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, we need a model with higher capacity for storing patterns than an MSOM, which is constrained to encode input patterns in relatively localist representations. But such networks are now readily available: for instance, the new generation of Hopfield networks perform the same kind of unsupervised associative learning as an MSOM, but with vastly higher capacity (see e.g., Krotov and Hopfield, 2016 , and especially Demircigil et al, 2017 ). Of course, the question of scalability remains to be addressed empirically—and this is something we plan to pursue in future work.…”

Section: Discussionmentioning

confidence: 99%

A Neural Network Model for Learning 3D Object Representations Through Haptic Exploration

2021

View full text Add to dashboard Cite

Humans initially learn about objects through the sense of touch, in a process called “haptic exploration.” In this paper, we present a neural network model of this learning process. The model implements two key assumptions. The first is that haptic exploration can be thought of as a type of navigation, where the exploring hand plays the role of an autonomous agent, and the explored object is this agent's “local environment.” In this scheme, the agent's movements are registered in the coordinate system of the hand, through slip sensors on the palm and fingers. Our second assumption is that the learning process rests heavily on a simple model of sequence learning, where frequently-encountered sequences of hand movements are encoded declaratively, as “chunks.” The geometry of the object being explored places constraints on possible movement sequences: our proposal is that representations of possible, or frequently-attested sequences implicitly encode the shape of the explored object, along with its haptic affordances. We evaluate our model in two ways. We assess how much information about the hand's actual location is conveyed by its internal representations of movement sequences. We also assess how effective the model's representations are in a reinforcement learning task, where the agent must learn how to reach a given location on an explored object. Both metrics validate the basic claims of the model. We also show that the model learns better if objects are asymmetrical, or contain tactile landmarks, or if the navigating hand is articulated, which further constrains the movement sequences supported by the explored object.

show abstract

Section: Discussionmentioning

confidence: 99%

A Neural Network Model for Learning 3D Object Representations Through Haptic Exploration

2021

View full text Add to dashboard Cite

show abstract

“…Furthermore, certain RCNNs can be viewed as kernelized versions of the Hopfield network with Hebbian learning [16,17,18]. Finally, RCNNs are closely related to the dense associative memory model introduced by Krotov and Hopfield to establish the duality between associative memories and deep learning [44,45].…”

Section: Quaternion-valued Recurrent Correlation Neural Networkmentioning

confidence: 99%

An Introduction to Quaternion-Valued Recurrent Projection Neural Networks

Valle

Lobo

2019

2019 8th Brazilian Conference on Intelligent Systems (BRACIS)

View full text Add to dashboard Cite

Hypercomplex-valued neural networks, including quaternion-valued neural networks, can treat multi-dimensional data as a single entity. In this paper, we present the quaternion-valued recurrent projection neural networks (QRPNNs). Briefly, QRPNNs are obtained by combining the non-local projection learning with the quaternion-valued recurrent correlation neural network (QRCNNs). We show that QRPNNs overcome the cross-talk problem of QRCNNs. Thus, they are appropriate to implement associative memories. Furthermore, computational experiments reveal that QRPNNs exhibit greater storage capacity and noise tolerance than their corresponding QRCNNs.

show abstract

“…In [30] it is proved that for ()exp() Fuu = this memory allows one to retrieve exp() ND a = randomly distorted vectors (within Ham dist/2 D < from the stored vectors) by a single step of the sequential dynamics, for some 0ln2/2 a << , depending on the distortion, with probability converging to 1 for D ®¥.…”

Section: The Generalization Of Krotov-hopfieldmentioning

confidence: 99%

Neural Autoassociative Memories for Binary Vectors: A Survey

Gritsenko¹,

Rachkovskij²,

Frolov³

et al. 2017

Kibern. vyčisl. teh.

View full text Add to dashboard Cite

Introduction. Neural network models of autoassociative, distributed memory allow storage and retrieval of many items (vectors) where the number of stored items can exceed the vector dimension (the number of neurons in the network). This opens the possibility of a sublinear time search (in the number of stored items) for approximate nearest neighbors among vectors of high dimension.The purpose of this paper is to review models of autoassociative, distributed memory that can be naturally implemented by neural networks (mainly with local learning rules and iterative dynamics based on information locally available to neurons).Scope. The survey is focused mainly on the networks of Hopfield, Willshaw and Potts, that have connections between pairs of neurons and operate on sparse binary vectors. We discuss not only autoassociative memory, but also the generalization properties of these networks. We also consider neural networks with higher-order connections and networks with a bipartite graph structure for non-binary data with linear constraints.Conclusions. In conclusion we discuss the relations to similarity search, advantages and drawbacks of these techniques, and topics for further research. An interesting and still not completely resolved question is whether neural autoassociative memories can search for approximate nearest neighbors faster than other index structures for similarity search, in particular for the case of very high dimensional vectors. Introduction. Neural network models of autoassociative, distributed memory allow storage and retrieval of many items (vectors) where the number of stored items can exceed the vector dimension (the number of neurons in the network). This opens the possibility of a sublinear time search (in the number of stored items) for approximate nearest neighbors among vectors of high dimension.The purpose of this paper is to review models of autoassociative, distributed memory that can be naturally implemented by neural networks (mainly with local learning rules and iterative dynamics based on information locally available to neurons).Scope. The survey is focused mainly on the networks of Hopfield, Willshaw and Potts, that have connections between pairs of neurons and operate on sparse binary vectors. We (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2017. № 2 (188) 6 discuss not only autoassociative memory, but also the generalization properties of these networks. We also consider neural networks with higher-order connections and networks with a bipartite graph structure for non-binary data with linear constraints.Conclusions. In conclusion we discuss the relations to similarity search, advantages and drawbacks of these techniques, and topics for further research. An interesting and still not completely resolved question is whether neural autoassociative memories can search for approximate nearest neighbors faster than other index structures for similarity search, in particular for the case of very high dimensional vectors. ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб...

show abstract

On a Model of Associative Memory with Huge Storage Capacity

Cited by 66 publications

References 18 publications

A Neural Network Model for Learning 3D Object Representations Through Haptic Exploration

A Neural Network Model for Learning 3D Object Representations Through Haptic Exploration

An Introduction to Quaternion-Valued Recurrent Projection Neural Networks

Neural Autoassociative Memories for Binary Vectors: A Survey

Contact Info

Product

Resources

About