Nonbinary Associative Memory With Exponential Pattern Retrieval Capacity and Iterative Learning

Salavati, Amir Hesam; Kumar, Krishan; Shokrollahi, Amin

doi:10.1109/tnnls.2013.2277608

Cited by 12 publications

(18 citation statements)

References 40 publications

(71 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several approaches have been proposed in the literature to extend the HN model, ranging from associative memories [71,56,37], to optimization problems [17,40,12] and system identification tasks [2]. The generalized HN models focused mainly on the output functions of neurons, which have been extended to have multiple inflection points.…”

Section: Hopfield Network With Neurons Partitioned Into Multiple Catmentioning

confidence: 99%

Learning node labels with multi-category Hopfield networks

Frasca

Bassis

Valentini

2015

Neural Comput & Applic

View full text Add to dashboard Cite

In several real-world node-label prediction problems on graphs, in fields ranging from computational biology to World-Wide-Web analysis, nodes can be partitioned into categories different from the classes to be predicted, on the basis of their characteristics or their common properties. Such partitions may provide further information about node classification that classical machine learning algorithms do not take into account. We introduce a novel family of parametric Hopfield networks (m-Category Hopfield Networks) and a novel algorithm (Hopfield Multi-Category -HoMCat), designed to appropriately exploit the presence of propertybased partitions of nodes into multiple categories. Moreover, the proposed model adopts a cost-sensitive learning strategy to prevent the remarkable decay in performance usually observed when instance labels are unbalanced, that is when one class of labels is highly under-represented than the other one. We validate the proposed model on both synthetic and real-world data, in the context of multi-species function prediction, where the classes to be predicted are the Gene Ontology terms and the categories the different species in the multi-species protein network. We carried out an intensive experimental validation, which on the one hand compares HoMCat with several state-of-the-art graph-based algorithms, and on the other hand reveals that exploiting meaningful prior partitions of input data can substantially improve classification performances.

show abstract

Section: Hopfield Network With Neurons Partitioned Into Multiple Catmentioning

confidence: 99%

Learning node labels with multi-category Hopfield networks

Frasca

Bassis

Valentini

2015

Neural Comput & Applic

View full text Add to dashboard Cite

show abstract

“…R is an important feature to consider since a code's dimensionality determines the dimensionality of it's null space, the object that is learned by the de-noising network. As discussed in [16], if we suppose that C ⊂ R n , and dim(C) = k < n, then there are n − k mutually orthogonal vectors that are also orthogonal to our code space (e.g. any basis for the null space of the code), each representing one valid constraint equation.…”

Section: Coding Theoretic Resultsmentioning

confidence: 99%

“…This and other de-noising processes are discussed in greater detail in [21] and [16]. Note that this de-noising mechanism differs from error correction methods presented in [19] and [26] in that information contributed by place cells only reaches grid cells through constraint neurons, and place information contributed by grid cells at module i only reaches other modules through constraint neurons if connectivity allows.…”

Section: De-noising and Decodingmentioning

confidence: 99%

On the Organization of Grid and Place Cells: Neural Denoising via Subspace Learning

Schwartz

Koyluoglu

2019

Neural Computation

View full text Add to dashboard Cite

Place cells in the hippocampus are active when an animal visits a certain location (referred to as a place field) within an environment. Grid cells in the medial entorhinal cortex (MEC) respond at multiple locations, with firing fields that form a periodic and hexagonal tiling of the environment. The joint activity of grid and place cell populations, as a function of location, forms a neural code for space. An ensemble of codes is generated by varying grid and place cell population parameters. For each code in this ensemble, codewords are generated by stimulating a network with a discrete set of locations. In this manuscript, we develop an understanding of the relationships between coding theoretic properties of these combined populations and code construction parameters. These relationships are revisited by measuring the performances of biologically realizable algorithms implemented by networks of place and grid cell populations, as well as constraint neurons, which perform de-noising operations. Objectives of this work include the investigation of coding theoretic limitations of the mammalian neural code for location and how communication between grid and place cell networks may improve the accuracy of each population's representation. Simulations demonstrate that de-noising mechanisms analyzed here can significantly improve fidelity of this neural representation of space. Further, patterns observed in connectivity of each population of simulated cells suggest that inter-hippocampalmedial-entorhinal-cortical connectivity decreases downward along the dorsoventral axis.

show abstract

“…Iterative algorithms for learning constraint matrix and vector recovery . In [78,145], they consider the problem of the exact retrieval (with high probability) of vectors that belong to a subspace of dimension less than D . The graded weights of the bipartite graph connections representing linear constraints are learned from the vectors of the base (which have only non-negative integer components).…”

Section: Nams With a Bipartite Graph Structure For Nonbinary Data Witmentioning

confidence: 99%

“…In [145], y from a subspace of dimension dD < are considered. Training forms a matrix W of Ddnon-zero linearly independent vectors orthogonal to the vectors y of the base: 0 = Wy for all y of the base.…”

Section: Nams With a Bipartite Graph Structure For Nonbinary Data Witmentioning

confidence: 99%

Neural Autoassociative Memories for Binary Vectors: A Survey

Gritsenko¹,

Rachkovskij²,

Frolov³

et al. 2017

Kibern. vyčisl. teh.

View full text Add to dashboard Cite

Introduction. Neural network models of autoassociative, distributed memory allow storage and retrieval of many items (vectors) where the number of stored items can exceed the vector dimension (the number of neurons in the network). This opens the possibility of a sublinear time search (in the number of stored items) for approximate nearest neighbors among vectors of high dimension.The purpose of this paper is to review models of autoassociative, distributed memory that can be naturally implemented by neural networks (mainly with local learning rules and iterative dynamics based on information locally available to neurons).Scope. The survey is focused mainly on the networks of Hopfield, Willshaw and Potts, that have connections between pairs of neurons and operate on sparse binary vectors. We discuss not only autoassociative memory, but also the generalization properties of these networks. We also consider neural networks with higher-order connections and networks with a bipartite graph structure for non-binary data with linear constraints.Conclusions. In conclusion we discuss the relations to similarity search, advantages and drawbacks of these techniques, and topics for further research. An interesting and still not completely resolved question is whether neural autoassociative memories can search for approximate nearest neighbors faster than other index structures for similarity search, in particular for the case of very high dimensional vectors. Introduction. Neural network models of autoassociative, distributed memory allow storage and retrieval of many items (vectors) where the number of stored items can exceed the vector dimension (the number of neurons in the network). This opens the possibility of a sublinear time search (in the number of stored items) for approximate nearest neighbors among vectors of high dimension.The purpose of this paper is to review models of autoassociative, distributed memory that can be naturally implemented by neural networks (mainly with local learning rules and iterative dynamics based on information locally available to neurons).Scope. The survey is focused mainly on the networks of Hopfield, Willshaw and Potts, that have connections between pairs of neurons and operate on sparse binary vectors. We (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2017. № 2 (188) 6 discuss not only autoassociative memory, but also the generalization properties of these networks. We also consider neural networks with higher-order connections and networks with a bipartite graph structure for non-binary data with linear constraints.Conclusions. In conclusion we discuss the relations to similarity search, advantages and drawbacks of these techniques, and topics for further research. An interesting and still not completely resolved question is whether neural autoassociative memories can search for approximate nearest neighbors faster than other index structures for similarity search, in particular for the case of very high dimensional vectors. ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб...

show abstract

Nonbinary Associative Memory With Exponential Pattern Retrieval Capacity and Iterative Learning

Cited by 12 publications

References 40 publications

Learning node labels with multi-category Hopfield networks

Learning node labels with multi-category Hopfield networks

On the Organization of Grid and Place Cells: Neural Denoising via Subspace Learning

Neural Autoassociative Memories for Binary Vectors: A Survey

Contact Info

Product

Resources

About