Connectionist semantic systematicity

Abstract

Fodor and Pylyshyn (1988) argue that connectionist models are not able to display systematicity other than by implementing a classical symbol system. This claim entails that connectionism cannot compete with the classical approach as an alternative architectural framework for human cognition.We present a connectionist model of sentence comprehension that does not implement a symbol system yet behaves systematically. It consists in a recurrent neural network that maps sentences describing situations in a microworld, onto representations of these situations. After being trained on particular sentences-situation pairs, the model can comprehend new sentences, even if these describe new situations. We argue that this systematicity arises robustly and in a psychologically plausible manner because it depends on structure inherent in the world.

Expectation-based Comprehension: Modeling the Interaction of World Knowledge and Linguistic Experience

“…Frank et al, 2009Frank et al, , 2003. It must be noted, however, that none of the presented machinery or results hinges upon this choice-it primarily serves the controlled investigation of the model's performance.…”

“…To this end, we employ a nondeterministic dimension selection algorithm (rather than a dimension reduction algorithm; cf. Frank et al, 2009): (1) Compute a vectorc m containing the comprehension score comprehensionða; bÞ for each combination of the n atomic propositions from the m  n space. (2) Randomly select k rows from the original m  n situation-state space.…”

“…In what follows, we show that DSS-derived situation vectors offer meaning representations that effectively instantiate situation models. We will first describe how a DSS can be constructed using a "microworld" approach, and show that the DSS-derived situation vectors are inherently compositional and probabilistic (Frank, Haselager, & van Rooij, 2009;Frank et al, 2003). Then, we will illustrate how comprehension in DSS is formalized as navigation through situation-state space.…”

Systematicity and Conceptual Pluralism

“…The early debate between Smolensky (1988Smolensky ( , 1991aSmolensky ( , 1991b and Fodor et al (Fodor and McLaughlin, 1990;Fodor, 1997) can be understood in those terms, and the gist of the dilemma posed by Fodor et al comes to this: if Smolensky is capable of showing that his models do have a constituent structure, then they are implementations of a classical system -given that they are based on the same relevant explanatory principles; if they do not have a constituent structure, then they cannot account for systematicity. The countless subsequent connectionist attempts of proving that this or that network has systematic capabilities -I will save space referring to Hadley (1994) for a review and criticism of early attempts, and to Frank et al (2009) for later ones-are subject, despite their differences, to basically the same sort of objection.…”

“…What I deny is that there is, as of today, an answer that satisfies most authors on the connectionist side, and this is enough to be at least suspicious that the challenge has been met. To put but one recent example, Frank et al (2009) review previous connectionist attempts to provide a model with semantic systematicity (Hadley, 1994) without implementing a classical system. They find all of them wanting only to propose their own model that, allegedly, succeeds in the task.…”