Gradiency and visual context in syntactic garden-paths

Abstract

Through recording the streaming x, y coordinates of computer-mouse movements, we report evidence that visual context provides an immediate constraint on the resolution of syntactic ambiguity in the visual-world paradigm. This finding converges with previous eye-tracking results that support a constraint-based account of sentence processing, in which multiple partially-active syntactic alternatives compete against one another with the help of not only syntactic, semantic, and statistical factors, but also nonlinguistic factors such as visual context. Eye-tracking results in the visual-world paradigm are consistent with theories that posit limited interaction between context and syntax, but they are still consistent with related theories that allow immediate interaction but require serial pursuit of syntactic structures, such as the unrestricted race model. To tease apart the constraint-based and unrestricted-race accounts of sentence processing, the distribution of computer-mouse trajectories was analyzed for evidence of two populations of trials: those where only the correct parse was pursued and those where only the incorrect parse was pursued. We found no evidence of bimodality in the distribution of trajectory curvatures. Simulations with a constraint-based model produced trajectories that matched the human data. A nonlinguistic control study demonstrated the mouse-tracking paradigm's ability to elicit bimodal distributions of trajectory curvatures in certain experimental conditions. With effects of context posing a challenge for syntax-first models, and the absence of bimodality in the distribution of garden-path magnitude posing a challenge for unrestricted-race models, these converging methods support the constraint-based theory's account that the reason diverse contextual factors are able to bias one or another parse at the point of ambiguity is because those syntactic alternatives are continually partially-active in parallel, not discretely winnowed. Keywords Syntax; Continuous; Dynamical; Language and Vision InteractionWhat exactly is a garden-path? About three decades ago, the term was introduced to the psycholinguistic literature in describing what it feels like to have been led astray by syntactic preferences while reading a sentence. The reader reaches some later portion of the sentence, where the syntax and/or the semantics are no longer commensurate or sensible with how she's been parsing the sentence up to that point, and she feels as though she's has been "led down the garden-path." But is a garden-path due to the discrete computation of a mental representation that turns out to be inappropriate and must then be deleted and replaced by an alternative mental representation-as seen with discrete computing algorithms (e.g., Budiu & Anderson, 2004;Dietrich & Markman, 2003;Newell, 1990)? Or is a garden-path due to multiple partially-active mental representations simultaneously competing with one anotherSend correspondence to: Thomas Farmer, Department of Psychology, Cornell University, Ithaca...

The interaction of visual and linguistic saliency during syntactic ambiguity resolution

“…We used a classic syntactic attachment ambiguity paradigm, which has been extensively used in psycholinguistics to uncover mechanisms of linguistic ambiguity resolution by studying attention in visual scenes (Spivey-Knowlton et al, 2002;Snedeker & Trueswell, 2003;Farmer, Anderson, & Spivey, 2007;Novick et al, 2008;Ferreira et al, 2013). In such a setup, visual and linguistic information can point either to the same or to a different regions in the display, hence promoting either the same or a different resolution of the ambiguity.…”

Assessing bimodality to detect the presence of a dual cognitive process

“…Thus, constraint-based accounts predict that certain response distributions will be unimodal. In particular, when syntax is ambiguous, these accounts predict that trials will reveal a single, continuous range of competition between the possible interpretations of the ambiguity, thereby giving way to a unimodal distribution over behavioral responses (Farmer, Anderson, & Spivey, 2007). A similar difference in predictions occurs in social categorization research.…”

Situated sentence processing: The coordinated interplay account and a neurobehavioral model

“…Implemented models, however, address only the high-level dynamics in which a set of pre-specified interpretations is activated (McRae et al, 1998;Tanenhaus et al, 2000;Spivey, 2007). Thus, even when such models include visuomotor constraints (Farmer, Anderson, & Spivey, 2007), they still shed little light on how the mechanisms of incremental sentence understanding interact with visual perceptual processes, and vice versa. Embodied accounts of language processing, while explaining the resonance that exists between language and visuomotor representations (Zwaan et al, 2002;Glenberg & Kaschak, 2002) as well as temporal aspects of the simulation of events (Claus & Kelter, 2006), have neither paid much attention to the compositional mechanisms of language comprehension and their time course (but see Glenberg & Robertson, 1999;Zwaan & Taylor, 2006) nor to the development of implementable computational models with broader linguistic coverage (see Crocker, 2005, for discussion).…”