Mental models, pictures, and text: Integration of spatial and verbal information

Abstract

In the past several years, there has been an acceleration in the publication of cognitive research on the interplay between linguistic and pictorial/spatial information. To report on and encourage this sort of research, we organized a symposium at the 1991 meeting of the Midwestern Psychological Association. The articles in this special section of Memory & Cognition are based on the work presented at the symposium. In this introduction, we offer a suggestion for why the integration of linguistic and spatial information is not only a possibility, but a requirement for effective communication. Our suggestion follows the linguistic analysis of the closed-class elements that convey spatial relations, the prepositions (Talmy, 1983). The structure of language provides but a small set of prepositions to encode the vast number of spatial relations that we can perceive. Thus, to understand a situation that a speaker or a writer is conveying, the listener or reader must combine linguistic information with (perhaps metric) spatial information derived from pictures, the environment, or memory.In 1991, at the meeting of the Midwestern Psychological Association, we held a symposium on the recent surge in research on the relation between language, visuospatial information, and visuospatial modes of communication (pictures, diagrams, maps, etc.). Heretofore, cognitive psychologists have seized on the nominal, surface differences between linguistic and visuospatial presentations as a useful distinction for guiding and circumscribing research on comprehension and memory. This strategy has produced an informative and rich literature on comprehension of connected discourse, and a somewhat less extensive literature on processing and memory for representational drawing, pictures, and maps. Yet the sufficiency of this distinction for creating a fruitful experimental enterprise may not be mimicked in the sufficiency of these modes for individually supporting effective communication.For instance, consider the assertion by Taylor and Tversky (1992a, p. 495) that "Language is a surrogate for experience. " If this were not so, it would be hard to understand how language could be used to inform us about events and objects with which we had no direct contact. Nonetheless, language can be distinctly inferior to ex- perience. Our perceptual apparatus delivers (usually) veridical, organized, detailed representations. Language may not do any of that. In particular, the structure of language is particularly ill-suited for communicating the spatial organization to which our perceptual apparatus seems tuned.The problem of language and space is revealed by an analysis of closed-elass (i.e., limited in size and hard to add to) grammatical terms used to indicate spatial relations. In English, only some SO-I00 prepositions are used to convey spatial relations (Jackendoff & Landau, 1991). These are words such as above, across, nearby, and upon. Clearly, however, our perceptual apparatus can make much finer metric distinctions. When we wish to use langu...

How did you get here from there? Verbal overshadowing of spatial mental models

“…This has long been a topic of some debate (e.g. Glenberg and McDaniel, 1992). Indeed, a number of researchers interested in text processing and mental models have investigated the construction of spatial mental models using different types of verbal descriptions as the independent measure (e.g.…”

Three Turtles in Danger: Spontaneous Construction of Causally Relevant Spatial Situation Models.

“…Alternatively, if a narrated event was not expected, causally relevant spatial relations may be represented because the recipient tries to construct an explanation for the event (Trabasso & Suh, 1993). In several studies that explored similar ideas, causal relevance proved to be crucial for spontaneous spatial representation under the labels functionality of spatial relations (Radvansky & Copeland, 2000;Radvansky, Copeland, & Zwaan, 2003), relevance for protagonist's goals (Levine & Klin, 2001;Morrow, Bower, & Greenspan, 1990), or required for comprehension (Glenberg & McDaniel, 1992;Hakala, 1999).…”

“…It is the prominent illustration of situation model construction by Bransford et al (1972, Experiment 1), who used a sentence-recognition paradigm (see also Garnham, 1981;Glenberg & McDaniel, 1992;Hakala, 1999;Radvansky & Copeland, 2001;Reyna & Kiernan, 1994;Rinck et al, 2001). Participants were instructed to listen carefully to sentences like 1A and 2A below and were told that they would be asked questions about these sentences afterward.…”

A Look into the Mind of the Negotiator: Mental Models in Negotiation

“…Mental models have been a focus of research on the representation and use of causal knowledge in many domains including naive physics (Forbus & Gentner, 1986;McClosky, 1983), astronomical knowledge (Vosniadou & Brewer, 1992), spatial representations (Forbus, 1983;Glenberg & McDaniel, 1992;McNarma, 1986), problem solving by analogy (Bassok, 1990;Gentner, Rattermann, & Forbus, 1993;Holyoak, 1984;Holyoak & Koh, 1987;Keane, 1988), physical mechanisms (Hegarty & Just, 1993;Kempton, 1986;Kieras & Bovair, 1984), the understanding of man-machine systems (Chee, 1993), judgments of probability ( Johnson-Laird, Legrenzi, Girotto, Legrenzi, & Caverni, 1999;Kahneman & Tversky, 1982), and propositional reasoning ( Johnson-Laird, Byrne, & Schaeken, 1992). Although the mental model concept has its share of critics (e.g.…”