Hardware Implementation of Associative Memories Based on Multiple-Valued Sparse Clustered Networks

Onizawa, Naoya; Jarollahi, Hooman; Hanyu, Takahiro; Gross, Warren J.

doi:10.1109/jetcas.2016.2528721

“…Since the information content of Willshaw-Potts vectors is low, the crit N is higher than for the Willshaw network. This network was rediscovered as the GB network in [58] with various modifications [3,59] and hardware implementations (for example, [117] with non-binary connections). The GB network is oriented for exact retrieval of vectors with distortion by deletion (columns without values activate all neurons).…”

Section: Willshaw-potts Networkmentioning

confidence: 99%

Neural Autoassociative Memories for Binary Vectors: A Survey

Gritsenko¹,

Rachkovskij²,

Frolov³

et al. 2017

Kibern. vyčisl. teh.

View full text Add to dashboard Cite

Introduction. Neural network models of autoassociative, distributed memory allow storage and retrieval of many items (vectors) where the number of stored items can exceed the vector dimension (the number of neurons in the network). This opens the possibility of a sublinear time search (in the number of stored items) for approximate nearest neighbors among vectors of high dimension.The purpose of this paper is to review models of autoassociative, distributed memory that can be naturally implemented by neural networks (mainly with local learning rules and iterative dynamics based on information locally available to neurons).Scope. The survey is focused mainly on the networks of Hopfield, Willshaw and Potts, that have connections between pairs of neurons and operate on sparse binary vectors. We discuss not only autoassociative memory, but also the generalization properties of these networks. We also consider neural networks with higher-order connections and networks with a bipartite graph structure for non-binary data with linear constraints.Conclusions. In conclusion we discuss the relations to similarity search, advantages and drawbacks of these techniques, and topics for further research. An interesting and still not completely resolved question is whether neural autoassociative memories can search for approximate nearest neighbors faster than other index structures for similarity search, in particular for the case of very high dimensional vectors. Introduction. Neural network models of autoassociative, distributed memory allow storage and retrieval of many items (vectors) where the number of stored items can exceed the vector dimension (the number of neurons in the network). This opens the possibility of a sublinear time search (in the number of stored items) for approximate nearest neighbors among vectors of high dimension.The purpose of this paper is to review models of autoassociative, distributed memory that can be naturally implemented by neural networks (mainly with local learning rules and iterative dynamics based on information locally available to neurons).Scope. The survey is focused mainly on the networks of Hopfield, Willshaw and Potts, that have connections between pairs of neurons and operate on sparse binary vectors. We (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2017. № 2 (188) 6 discuss not only autoassociative memory, but also the generalization properties of these networks. We also consider neural networks with higher-order connections and networks with a bipartite graph structure for non-binary data with linear constraints.Conclusions. In conclusion we discuss the relations to similarity search, advantages and drawbacks of these techniques, and topics for further research. An interesting and still not completely resolved question is whether neural autoassociative memories can search for approximate nearest neighbors faster than other index structures for similarity search, in particular for the case of very high dimensional vectors. ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб...

show abstract

“…Since the information content of Willshaw-Potts vectors is low, the crit N is higher than for the Willshaw network. This network was rediscovered as the GB network in [58] with various modifications [3,59] and hardware implementations (for example, [117] with non-binary connections). The GB network is oriented for exact retrieval of vectors with distortion by deletion (columns without values activate all neurons).…”

Section: Potts Namsmentioning

confidence: 99%

Untitled

2017

Kibern. vyčisl. teh.

0

View full text Add to dashboard Cite

Introduction. Neural network models of autoassociative, distributed memory allow storage and retrieval of many items (vectors) where the number of stored items can exceed the vector dimension (the number of neurons in the network). This opens the possibility of a sublinear time search (in the number of stored items) for approximate nearest neighbors among vectors of high dimension. The purpose of this paper is to review models of autoassociative, distributed memory that can be naturally implemented by neural networks (mainly with local learning rules and iterative dynamics based on information locally available to neurons). Scope. The survey is focused mainly on the networks of Hopfield, Willshaw and Potts, that have connections between pairs of neurons and operate on sparse binary vectors. We 5

show abstract

“…Secondly, the robustness of the centralized algorithm is poor compared with the distributed one under the scenario of single-point failure. Thirdly, the distributed algorithm has good scalability for easy implementation of plugging-in and plugging-out of the micro unit clusters [23], which is also easy to be maintained under a lower operation cost.…”

Section: Introductionmentioning

confidence: 99%

Coordinated Active Power Dispatch for a Microgrid via Distributed Lambda Iteration

Hu

¹

,

Chen

²

,

Cao

³

et al. 2017

IEEE J. Emerg. Sel. Topics Circuits Syst.

View full text Add to dashboard Cite

A novel distributed optimal dispatch algorithm is proposed for coordinating the operation of multiple micro units in a microgrid, which has incorporated the distributed consensus algorithm in multi-agent systems and the λ-iteration optimization algorithm in economic dispatch of power systems. Specifically, the proposed algorithm considers the global active power constraint by adding a virtual pinner and it can deal with the optimization problem with any initial states. That is, it can realize the global optimization and avoid the defect of the initial conditions' sensitivity in the optimization problem. On the other hand, the proposed optimization algorithm can either be used for off-line calculation or be utilized for on-line operation and has the ability to survive single-point failures and shows good robustness in the iteration process. Numerical studies in a seven bus microgrid demonstrate the effectiveness of the proposed algorithm.

show abstract

Hardware Implementation of Associative Memories Based on Multiple-Valued Sparse Clustered Networks

Cited by 5 publications

References 27 publications

Neural Autoassociative Memories for Binary Vectors: A Survey

Neural Autoassociative Memories for Binary Vectors: A Survey

Untitled

Coordinated Active Power Dispatch for a Microgrid via Distributed Lambda Iteration

Contact Info

Product

Resources

About