Distributed Vector Representations of Words in the Sigma Cognitive Architecture

The Sigma Cognitive Architecture and System: Towards Functionally Elegant Grand Unification

“…Since 2008, when the development of Sigma was begun, progress towards grand unification has occurred across learning and memory (Rosenbloom, 2010(Rosenbloom, , 2012a(Rosenbloom, , 2014Rosenbloom et al, 2013;Pynadath et al, 2014;Ustun et al, 2014;, perception and imagery (Chen et al, 2011;Rosenbloom, 2011bRosenbloom, , 2012bJoshi et al, 2014), reasoning and problem solving (Chen et al, 2011;Rosenbloom, 2011c), speech and language (Joshi et al, 2014;Rosenbloom et al, 2013;Ustun et al, 2014), and social and affective cognition (Pynadath et al, 2013(Pynadath et al, , 2014Rosenbloom, Gratch and Ustun, 2015). Insights from both natural and artificial cognition have been incorporated, and some novel contributions to cognitive modeling -such as the emergence of base-level activation as the prior learned over a temporal variable in episodic memory (Rosenbloom, 2014) -have started to appear.…”

“…Such vectors are typically used, for example, to learn and represent the meaning of words or concepts in a form that enables vector similarity -for example, as computed by the standard cosine similarity metric -to correlate with meaning similarity. We have initial experience in learning and using such vectors in Sigma -under the name DVRS (Distributed Vector Representation in Sigma) -for example, in supporting analogical reasoning among words (Ustun et al, 2014). Distributed vectors do require negative functional values, as shown in the figure, so as to exploit all four quadrants of the vector space.…”

“…The last idiom that will be covered in this subsection is for distributed vector memory (Figure 13) and its associated reasoning (Ustun et al, 2014). This idiom is like the one for imagery memory in only constraining the predicates.…”

“…With these changes, vector summation occurs via action combination, vector binding continues to occur via multiplication, and vector comparison occurs via cosine similarity (based on a combination of multiplication, summarization and vector normalization). Figure 77, for example, illustrates how knowledge about which other words have co-occurred with a particular word in a fragment of text is converted into a single new vector representing this context (Ustun et al, 2014). It starts with (a) a local Boolean vector over the vocabulary that has a 1 for each co-occurring word and a 0 for all other words, and (b) the lexicon, represented as a matrix that maps local word representations into distributed representations.…”

“…Because of the large number of large distributed vectors used here, this idiom presently runs rather slowly in Sigma. We are investigating ways to speed this up in Sigma, but in the meantime a simulator, DVRS', has been used to generate results not currently feasible within Sigma (Ustun et al, 2014). Table 5 illustrates the results of this simulator for the task of finding similar words.…”