A Simulation of Visual Imagery*

Abstract

This paper describes an operational computer simulation of visual mental imagery in humans. The structure of the simulation was motivated by results of experiments on,how people represent information in. and access information from; visual images. The simulation includes a "surface representation." which is spatial and quasi-pictorial, and an underlying "deep representation." which contains "perceptual" information encoding appearance plus "propositional" information describing facts about an object. The simulation embodies a theory of how surface images are generated from deep representations. and how surface images are pmcessed when one accesses information embedded in them. The simulation also offers an account of various sorts of imagery transformations.Our simulation is a model of how people represent information in, and later retrieve information from, visual .mental images. That is, whe'n retrieving some fact, people sometimes report generating an image and inspecting it; for example. if asked what shape are a beagle's ears, one may recall this information by mentally picturing the dog's head, and "looking" at the ears. We wanted to understand how such processes operate, and felt that a simulation approach was useful for a number of reasons: First. it allows explicit modeling of a system of complex processes. Second. it provides a "sufficiency proof," a demonstration that one's model is in fact adequate to account for some range of data, if the program runs as claimed. Third, we felt that by attempting to motivate construction of the model with actual data about human cognition, we would be confronted by important empirical issues that may otherwise have been overlooked; we also felt that such an approach would lead us to collect a set of data that would build, that would accumulate to form a "big picture" (cf. Newell. 1973; see also Kosslyn. in press).

The Relationship Between Bilingualism and the Performance of Spatial Tasks

“…Theories of imagery emerged at this time (for example Kosslyn & Shwartz, 1977;Paivio, 1971). Paivio's (1971) dual coding theory suggests that the memory stores information both pictorially and verbally.…”

“…The simulation experiments of Kosslyn and Shwartz (1977), and the clinical experiments of Farah et al (1988) and Kosslyn (1987), investigated brain activity during the performance of imagery tasks. A focus of much of the research was to clarify the debate about the nature of a mental image, whether mental images are pictorial or propositional.…”

“…Paivio (1971), who allows for both approaches, claims that the pictorial system is better for spatial processing whereas the verbal system is more useful for serial or sequential processing (remembering the order of objects). Kosslyn and Shwartz (1977) describe the existence of not only a quasipictorial 'surface representation' or general appearance of an image, but also a 'deep representation' which contains pictorial information as well as additional facts about an image. They were able to simulate the process of visual imagery by computer and conclude that image representation is built up from a diversity of information stored in the brain.…”

“…In manipulating the topological structures used in this study, the data encoded spatial arrays as a means to constructing an image described by Kosslyn and Shwartz (1977) may be effective and thus could play a key role in the solution of these tasks.…”

The Mental Image in Cartographic Communication

“…may have to mentally image the car in order to retrieve the information. 28 This same example can be put into geographic terms. For example, the question of whether Houston and San Antonio are both in the state of Texas may be most quickly answered by consulting a propositional file.…”