A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

Kleyko, Denis; Rachkovskij, Dmitri A.; Osipov, Evgeny; Rahimi, Abbas

doi:10.48550/arxiv.2112.15424

Cited by 6 publications

(9 citation statements)

References 89 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We begin by showing numerical results which measure the quality of the retrieved structures in terms of the unbinding error P ϵ and the SNR of overlaps defined in Eqn. 20. In all reported results, the extracted item (and the associated query) comes from pairs that are not part of the cueing structure S 0 .…”

Section: Retrieval Of Structured Memoriesmentioning

confidence: 96%

“…An early attempt used the tensor product to create a distributed representation of pairwise relations between discrete items [6]. Subsequently, several Vector-Symbolic Architectures (VSA) were proposed as compressions of the tensor product to avoid the increase in dimensionality of the representation, allowing for the creation of hierarchies of relations in a compact way [17][18][19][20][21][22]. More recently, several architectures for deep or recurrent neural networks have been proposed to promote flexible relational reasoning [23][24][25][26][27][28][29].…”

mentioning

confidence: 99%

See 1 more Smart Citation

Associative memory of structured knowledge

Steinberg

Sompolinsky

2022

Preprint

View full text Add to dashboard Cite

A long standing challenge in biological and artificial intelligence is to understand how new knowledge can be constructed from known building blocks in a way that is amenable for computation by neuronal circuits. Here we focus on the task of storage and recall of structured knowledge in long-term memory. Specifically, we ask how recurrent neuronal networks can store and retrieve multiple knowledge structures. We model each structure as a set of binary relations between events and attributes (attributes may represent e.g., temporal order, spatial location, role in semantic structure), and map each structure to a distributed neuronal activity pattern using a vector symbolic architecture (VSA) scheme. We then use associative memory plasticity rules to store the binarized patterns as fixed points in a recurrent network. By a combination of signal-to-noise analysis and numerical simulations, we demonstrate that our model allows for efficient storage of these knowledge structures, such that the memorized structures as well as their individual building blocks (e.g., events and attributes) can be subsequently retrieved from partial retrieving cues. We show that long-term memory of structured knowledge relies on a new principle of computation beyond the memory basins. Finally, we show that our model can be extended to store sequences of memories as single attractors.

show abstract

Section: Retrieval Of Structured Memoriesmentioning

confidence: 96%

mentioning

confidence: 99%

Associative memory of structured knowledge

Steinberg

Sompolinsky

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Hyperdimensional computing (HDC [1]), also known as Vector-Symbolic Architectures (VSA [2]) tries to combine the advantages of symbolic structured data representations and connectionist distributed vector representations. There is number of applications at the intersection of electrical engineering, machine learning, and cognitive computing, where HDC/VSA have demonstrated to be a promising approach (see, e.g., [3]- [13]).…”

Section: Introductionmentioning

confidence: 99%

Recursive Binding for Similarity-Preserving Hypervector Representations of Sequences

Rachkovskij¹,

Kleyko²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Hyperdimensional computing (HDC), also known as vector symbolic architectures (VSA), is a computing framework used within artificial intelligence and cognitive computing that operates with distributed vector representations of large fixed dimensionality. A critical step for designing the HDC/VSA solutions is to obtain such representations from the input data. Here, we focus on sequences and propose their transformation to distributed representations that both preserve the similarity of identical sequence elements at nearby positions and are equivariant to the sequence shift. These properties are enabled by forming representations of sequence positions using recursive binding and superposition operations. The proposed transformation was experimentally investigated with symbolic strings used for modeling human perception of word similarity. The obtained results are on a par with more sophisticated approaches from the literature. The proposed transformation was designed for the HDC/VSA model known as Fourier Holographic Reduced Representations. However, it can be adapted to some other HDC/VSA models.

show abstract

“…Therefore, it is important to emphasize that this article is focused heavily on the results obtained from a single research group and, hence, it does not give the full credit to the related ideas and methods developed by other groups. We highlight some connections to Hyperdimensional Computing (HDC) and Vector Symbolic Architectures (VSA), as well as to brain research, however, for a comprehensive treatment of HDC/VSA and its connection to APNNs the readers are kindly referred to [7,8].…”

Section: Introductionmentioning

confidence: 99%

Neural Distributed Representations for Artificial Intelligence and Modeling of Thinking

Rachkovskij¹,

Gritsenko²,

Volkov³

et al. 2022

Kibern. vyčisl. teh.

View full text Add to dashboard Cite

coming these shortcomings is a necessary condition for advancing from specialized Artificial Intelligence to general one, which requires the development of alternative approaches.The purpose of the paper is to present an overview of research in this direction, which has been carried out at the International Center for 25 years. The approach being developed stems from the ideas of N. M. Amosov and his scientific school. Connections to the Hyperdimensional Computing (HDC) and Vector Symbolic Architectures (VSA) field as well as to current brain research are also provided.Results. The concept of distributed data representation is outlined, including HDC/VSA that are capable of representing various data structures. The developed paradigm of Associative-Projective Neural Networks is considered: codevector representation of data, superposition and binding operations, general architecture, transformation of data of various types into codevectors, methods for solving problems and applications. Conclusion. An adequate representation of data is one of the key issues within the Artificial Intelligence. The main area of research reviewed in this article is the problem of representing heterogeneous data in Artificial Intelligence systems in a unified format based on modeling the neural organization of the brain and the mechanisms of thinking. The approach under development is based on the hypothesis of distributed representation of information in the brain and allows representing various types of data, from numeric values to graphs, as vectors of large but fixed dimensionality.The most important advantages of the developed approach are the possibility of natural integration and efficient processing of various types of data and knowledge, a high degree of parallel computing, reliability and resistance to noise, the possibility of hardware implementation with high performance and energy efficiency, data processing based on associative similarity search -similar to how human memory works. This allows one to unify the methods, algorithms, and software and hardware for Artificial Intelligence systems, increase their scalability in terms of speed and memory with an increase in data volume and complexity.The research creates the basis for overcoming the shortcomings of current approaches to the specialized Artificial Intelligence based on Deep Neural Networks and paves the way for the creation of Artificial General Intelligence.

show abstract

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

Cited by 6 publications

References 89 publications

Associative memory of structured knowledge

Associative memory of structured knowledge

Recursive Binding for Similarity-Preserving Hypervector Representations of Sequences

Neural Distributed Representations for Artificial Intelligence and Modeling of Thinking

Contact Info

Product

Resources

About