Vector space architecture for emergent interoperability of systems by learning from demonstration

Emruli, Blerim; Sandin, Fredrik; Delsing, Jerker

doi:10.1016/j.bica.2014.11.015

Cited by 6 publications

(7 citation statements)

References 74 publications

(122 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The alternative method that we used is an associative memory for vectors, specifically variations of the Sparse Distributed Memory (Kanerva, 1988 , 1993 ). Although the idea that the superposition vector shall be used as a working memory while the Sparse Distributed Memory is suitable to implement the long-term memory has been expressed previously, e.g., by Emruli et al ( 2015 ), no studies have been done to quantitatively compare these two alternatives. Also in a vein similar to this study, Steinberg and Sompolinsky ( 2022 ) examined how sets of key-value pairs represented with HD computing can be stored in associative memories using a Hopfield network.…”

Section: Discussionmentioning

confidence: 99%

On separating long- and short-term memories in hyperdimensional computing

et al. 2023

View full text Add to dashboard Cite

Operations on high-dimensional, fixed-width vectors can be used to distribute information from several vectors over a single vector of the same width. For example, a set of key-value pairs can be encoded into a single vector with multiplication and addition of the corresponding key and value vectors: the keys are bound to their values with component-wise multiplication, and the key-value pairs are combined into a single superposition vector with component-wise addition. The superposition vector is, thus, a memory which can then be queried for the value of any of the keys, but the result of the query is approximate. The exact vector is retrieved from a codebook (a.k.a. item memory), which contains vectors defined in the system. To perform these operations, the item memory vectors and the superposition vector must be the same width. Increasing the capacity of the memory requires increasing the width of the superposition and item memory vectors. In this article, we demonstrate that in a regime where many (e.g., 1,000 or more) key-value pairs are stored, an associative memory which maps key vectors to value vectors requires less memory and less computing to obtain the same reliability of storage as a superposition vector. These advantages are obtained because the number of storage locations in an associate memory can be increased without increasing the width of the vectors in the item memory. An associative memory would not replace a superposition vector as a medium of storage, but could augment it, because data recalled from an associative memory could be used in algorithms that use a superposition vector. This would be analogous to how human working memory (which stores about seven items) uses information recalled from long-term memory (which is much larger than the working memory). We demonstrate the advantages of an associative memory experimentally using the storage of large finite-state automata, which could model the storage and recall of state-dependent behavior by brains.

show abstract

Section: Discussionmentioning

confidence: 99%

On separating long- and short-term memories in hyperdimensional computing

et al. 2023

View full text Add to dashboard Cite

show abstract

“…An early example of concept-based learning and reasoning for SoSs automation is presented in [68], where concepts are represented in a vector-symbolic architecture and actions are automated using a combination of causality-based imitation learning and analogy making.…”

Section: Concept-based Learning and Reasoningmentioning

confidence: 99%

AI Concepts for System of Systems Dynamic Interoperability

Nilsson,

Javed,

Albertsson

et al. 2024

Sensors

View full text Add to dashboard Cite

Interoperability is a central problem in digitization and sos engineering, which concerns the capacity of systems to exchange information and cooperate. The task to dynamically establish interoperability between heterogeneous cps at run-time is a challenging problem. Different aspects of the interoperability problem have been studied in fields such as sos, neural translation, and agent-based systems, but there are no unifying solutions beyond domain-specific standardization efforts. The problem is complicated by the uncertain and variable relations between physical processes and human-centric symbols, which result from, e.g., latent physical degrees of freedom, maintenance, re-configurations, and software updates. Therefore, we surveyed the literature for concepts and methods needed to automatically establish sos with purposeful cps communication, focusing on machine learning and connecting approaches that are not integrated in the present literature. Here, we summarize recent developments relevant to the dynamic interoperability problem, such as representation learning for ontology alignment and inference on heterogeneous linked data; neural networks for transcoding of text and code; concept learning-based reasoning; and emergent communication. We find that there has been a recent interest in deep learning approaches to establishing communication under different assumptions about the environment, language, and nature of the communicating entities. Furthermore, we present examples of architectures and discuss open problems associated with ai-enabled solutions in relation to sos interoperability requirements. Although these developments open new avenues for research, there are still no examples that bridge the concepts necessary to establish dynamic interoperability in complex sos, and realistic testbeds are needed.

show abstract

“…It was shown to mimic learning in honey bees using the transformation as in [178,182]. In [77], an HDC/VSA-based approach for learning behaviors, based on observing and associating sensory and motor/action data represented by HVs, was proposed. A potential shortcoming of the approaches to learning holistic transformations presented above is that the objects/relations are assumed to be dissimilar to each other.…”

Section: Learning Holistic Transformations From Examplesmentioning

confidence: 99%

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

et al. 2023

View full text Add to dashboard Cite

This is Part II of the two-part comprehensive survey devoted to a computing framework most commonly known under the names Hyperdimensional Computing and Vector Symbolic Architectures (HDC/VSA). Both names refer to a family of computational models that use high-dimensional distributed representations and rely on the algebraic properties of their key operations to incorporate the advantages of structured symbolic representations and vector distributed representations. Holographic Reduced Representations [322, 327] is an influential HDC/VSA model that is well-known in the machine learning domain and often used to refer to the whole family. However, for the sake of consistency, we use HDC/VSA to refer to the field. Part I of this survey [223] covered foundational aspects of the field, such as the historical context leading to the development of HDC/VSA, key elements of any HDC/VSA model, known HDC/VSA models, and the transformation of input data of various types into high-dimensional vectors suitable for HDC/VSA. This second part surveys existing applications, the role of HDC/VSA in cognitive computing and architectures, as well as directions for future work. Most of the applications lie within the Machine Learning/Artificial Intelligence domain, however, we also cover other applications to provide a complete picture. The survey is written to be useful for both newcomers and practitioners.

show abstract

Vector space architecture for emergent interoperability of systems by learning from demonstration

Cited by 6 publications

References 74 publications

On separating long- and short-term memories in hyperdimensional computing

On separating long- and short-term memories in hyperdimensional computing

AI Concepts for System of Systems Dynamic Interoperability

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

Contact Info

Product

Resources

About