Holographic Declarative Memory: Distributional semantics as the architecture of memory

Abstract

We demonstrate that the key components of cognitive architectures—declarative and procedural memory—and their key capabilities—learning, memory retrieval, judgement, and decision-making—can be implemented as algebraic operations on vectors in a high-dimensional space. High-dimensional vector spaces underlie the success of modern machine learning techniques based on neural networks and deep learning. However, while neural networks have an impressive ability to process data to find patterns, they do not typically model high-level cognition, and it is often unclear how they work. Traditional, symbolic cognitive architectures can capture the complexities of high-level cognition and provide human-readable, explainable models, but have limited ability to detect patterns or learn. Vector-symbolic architectures, where symbols are represented as vectors, bridge the gap between the two approaches. We posit that cognitive architectures, if implemented in a vector-space model, represent a useful, explanatory model of the internal representations of otherwise opaque neural architectures. Our vector-space model accounts for primacy and recency effects in free recall, the fan effect in recognition, probability judgements, and human performance on an iterated decision task. Our model provides a flexible, scalable alternative to symbolic cognitive architectures at a level of description that bridges symbolic, quantum, and neural models of cognition.

Indirect associations in learning semantic and syntactic lexical relationships

“…While BEAGLE is a model of the mental lexicon, Dynamically Structured Holographic Memory (Rutledge-Taylor et al, 2014) is a variant of BEA-GLE applied to non-linguistic memory and learning tasks, such as learning sequences of actions for strategic game play. Kelly et al (2015) and Kelly and Reitter (2017) propose another BEAGLE variant, Holographic…”

How language processing can shape a common model of cognition

“…Thus modelling language acquisition forces us to adopt a cognitive architecture that uses a lossy memory model capable of generalization. Possible candidates include vector symbolic memory, which can replicate the behaviour of the ACT-R declarative memory but is scalable to language learning [19], and neural network models.…”

“…The choice between high-dimensional neural models of memory versus ACT-R leaves us with a trade-off between (in the case of neural models) a black box that fits large-scale data better and (in the case of ACT-R) transparent representations that can be interpreted by the scientist but account for large-scale language data less well. Models that use vector-symbolic architectures [12], such as BEAGLE [17,20], provide a middle ground of systems that are more interpretable than conventional neural networks and more scalable the traditional symbolic models like ACT-R [19].…”

Towards a Predictive Processing Implementation of the Common Model of Cognition

“…We propose a cognitive architecture that is built on two biologically plausible, neural models, namely a variant of predictive processing known as Neural Generative Coding (NGC) [27] and holographic memory [13]. Desirably, the use of these particular building blocks yields naturally scalable, local update rules (based on variants of Hebbian learning [11]) to adjust the overall system's synaptic weight parameters while facilitating robustness in acquiring, storing, and composing distributed representations of tasks encountered sequentially.…”