Semantic Systematicity in Connectionist Language Production

Calvillo, Jesús; Brouwer, Harm; Crocker, Matthew W.

doi:10.3390/info12080329

Cited by 4 publications

(4 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…John & McClelland, 1990). Our goal thus differs from modeling efforts that explicitly attempt to map linguistic input onto situation knowledge (e.g., Calvillo, Brouwer, & Crocker, 2016). We stress this point because we use terminology such as agent, patient, and instrument in describing activities that is also used to describe linguistic constructs.…”

Section: Architecturementioning

confidence: 99%

“…Each unit represents a concept. Localist representations are useful for expository purposes, although the learning algorithm can be used with distributed representations as well (Calvillo et al, 2016;Frank, Koppen, Noordman, & Vonk, 2003). The architecture in Figure 1 was used in all simulations reported herein, although simulations differ in their specific participants, processes, and contexts.…”

Section: Architecturementioning

confidence: 99%

See 1 more Smart Citation

A model of event knowledge.

Elman¹,

Masamune²

2019

Psychological Review

View full text Add to dashboard Cite

Our knowledge of events and situations in the world plays a critical role in our ability to understand what is happening around us, to predict what might happen next, and to comprehend language. What has not been so clear is the form and structure of this knowledge, how it is learned, and how it is deployed in real time. Despite many important theoretical proposals, often using different terminology such as schemas, scripts, frames, and event knowledge, developing a model that addresses these three questions (the form, learning, and use of such knowledge) has remained an elusive challenge for decades. In this article, we present a connectionist model of event knowledge that attempts to fill this gap. From sequences of activities, the model learns both the internal structure of activities as well as the temporal structure that organizes activity sequences. The model simulates a wide range of human behaviors that have been argued to involve the use of event knowledge and the temporal structure of events. Furthermore, it makes testable predictions about behaviors not yet observed. Most importantly, the model’s ability to learn event structure from experience is a novel solution to the question, “What is the form and representation of event knowledge?”

show abstract

Section: Architecturementioning

confidence: 99%

Section: Architecturementioning

confidence: 99%

A model of event knowledge.

Elman¹,

Masamune²

2019

Psychological Review

View full text Add to dashboard Cite

show abstract

“…For instance, a neural network model of language comprehension, similar to the one presented above, but employing meaning representations derived from an earlier formulation of the DFS framework (see [25]), has been used to successfully model the interaction between linguistic experience and world knowledge in comprehension [28]. Moreover, models employing such meaning representations have been shown to naturally capture inference and quantification [31], and generalize to unseen sentences and semantics, in both comprehension [31] and production [53]. Here, we have extended these results by showing how they capture phenomena such as negation, presupposition, and anaphoricity.…”

Section: Dfs In Cognitive Models Of Language Processingmentioning

confidence: 99%

A Framework for Distributional Formal Semantics

Venhuizen

Hendriks

Crocker

et al. 2019

Logic, Language, Information, and Computation

Self Cite

View full text Add to dashboard Cite

Natural language semantics has recently sought to combine the complementary strengths of formal and distributional approaches to meaning. More specifically, proposals have been put forward to augment formal semantic machinery with distributional meaning representations, thereby introducing the notion of semantic similarity into formal semantics, or to define distributional systems that aim to incorporate formal notions such as entailment and compositionality. However, given the fundamentally different 'representational currency' underlying formal and distributional approaches-models of the world versus linguistic co-occurrence-their unification has proven extremely difficult.Here, we define a Distributional Formal Semantics that integrates distributionality into a formal semantic system on the level of formal models. This approach offers probabilistic, distributed meaning representations that are also inherently compositional, and that naturally capture fundamental semantic notions such as quantification and entailment. Furthermore, we show how the probabilistic nature of these representations allows for probabilistic inference, and how the information-theoretic notion of "information" (measured in terms of Entropy and Surprisal) naturally follows from it. Finally, we illustrate how meaning representations can be derived incrementally from linguistic input using a recurrent neural network model, and how the resultant incremental semantic construction procedure intuitively captures key semantic phenomena, including negation, presupposition, and anaphoricity.

show abstract

“…There has been substantially less work on models of sentence production than sentence comprehension. It may seem straightforward to construct a production model by running a sentence comprehension model backwards, and this is indeed how two recent connectionist models of production were developed (Calvillo, Brouwer, & Crocker, 2016;Hinaut et al, 2015). However, the most successful and empirically validated sentence production models were specifically designed to simulate production.…”

Section: Sentence Productionmentioning

confidence: 99%

Neural Network Models of Language Acquisition and Processing

Frank¹,

Monaghan²,

Tsoukala³

2019

Human Language

View full text Add to dashboard Cite

Artificial neural network models (also known as Parallel Distributed Processing or Connectionist models) have been highly influential in cognitive science since the mid-1980s. The original inspiration for these systems comes from information processing in the brain, which emerges from a large number of (nearly) identical, simple processing units (neurons) that are interconnected into a network. Each unit receives activation from other units or by stimulation from the external world, and generates an output activation that is a function of the total input activation received. The unit then feeds the output activation onward to the units to which it is connected. Information processing is thus implemented in terms of activation flowing through this network.Each connection between two units has a weight that determines how strongly the first unit affects the second. These weights can be adapted, which constitutes learning, or "training" as it is commonly known in the neural network literature. Algorithms for network training can be roughly divided into supervised and unsupervised methods. Supervised training is applied when a specific and known input-to-output mapping is required (e.g., learning to transform orthographic to phonological representations). To accomplish this, the network is provided with a representative set of "training examples" of inputs and the corresponding target outputs. It then processes each example and the difference between the networks' actual output and the target output leads to an update of the connection weights such that, next time, the output error will be smaller. By far the best known and most used method for supervised training is the Backpropagation algorithm (Rumelhart, Hinton, & Williams, 1986) that makes the network's output activations for the training examples gradually converge toward the target outputs. Unsupervised training, in contrast, makes the network adapt to (aspects of) the statistical structure of input examples without mapping to target outputs (e.g., discovery of regularities in the phonological structure of language). These networks are well-suited to uncovering statistical structure present in the environment without requiring the modeller being aware what the structure is. One well-known example of an unsupervised training method is the learning rule proposed by Hebb (1949): Strengthen connections between units that are simultaneously active and weaken the connections between two units if only one is active.In spite of the superficial similarities between artificial and biological neural networks (i.e., interconnectivity and stimulation passing between neurons to determine their activation, and learning by adaptation of connection strengths), these cognitive models are not usually claimed to simulate processing at the level of biological neurons. Rather, neural network models form a description at Marr's (1982) algorithmic level, that is, they specify cognitive representations and operations while ignoring the biological implementation.Neural networks underwe...

show abstract

Semantic Systematicity in Connectionist Language Production

Cited by 4 publications

References 50 publications

A model of event knowledge.

A model of event knowledge.

A Framework for Distributional Formal Semantics

Neural Network Models of Language Acquisition and Processing

Contact Info

Product

Resources

About