Perceptual symbol systems

Abstract

Prior to the twentieth century, theories of knowledge were inherently perceptual. Since then, developments in logic, statistics, and programming languages have inspired amodal theories that rest on principles fundamentally different from those underlying perception. In addition, perceptual approaches have become widely viewed as untenable because they are assumed to implement recording systems, not conceptual systems. A perceptual theory of knowledge is developed here in the context of current cognitive science and neuroscience. During perceptual experience, association areas in the brain capture bottom-up patterns of activation in sensory-motor areas. Later, in a top-down manner, association areas partially reactivate sensory-motor areas to implement perceptual symbols. The storage and reactivation of perceptual symbols operates at the level of perceptual components – not at the level of holistic perceptual experiences. Through the use of selective attention, schematic representations of perceptual components are extracted from experience and stored in memory (e.g., individual memories of green, purr, hot). As memories of the same component become organized around a common frame, they implement a simulator that produces limitless simulations of the component (e.g., simulations of purr). Not only do such simulators develop for aspects of sensory experience, they also develop for aspects of proprioception (e.g., lift,run) and introspection (e.g., compare,memory,happy, hungry). Once established, these simulators implement a basic conceptual system that represents types, supports categorization, and produces categorical inferences. These simulators further support productivity, propositions, and abstract concepts, thereby implementing a fully functional conceptual system. Productivity results from integrating simulators combinatorially and recursively to produce complex simulations. Propositions result from binding simulators to perceived individuals to represent type-token relations. Abstract concepts are grounded in complex simulations of combined physical and introspective events. Thus, a perceptual theory of knowledge can implement a fully functional conceptual system while avoiding problems associated with amodal symbol systems. Implications for cognition, neuroscience, evolution, development, and artificial intelligence are explored.

Priming the mental time-line: effects of modality and processing mode

“…In contrast to theories which suppose that representations are coded in abstract and amodal ''concept nodes'' within semantic memory (e.g., Anderson 1983;Kintsch 1998), embodied theories assume that internal representations include multimodal perceptual and motor experiences from the outer world (Barsalou 1999;Glenberg 1997). These embodied theories receive support from the observation that the duration of mental operations in mental rotation tasks or mental migration tasks is related to the duration of real-world operations (Kosslyn 1975(Kosslyn , 1980Shepard and Metzler 1971) and that cognitive states interact with body states and body postures (e.g., Harmon-Jones and Peterson 2009;Strack et al 1988;Wells and Petty 1980).…”

Generalizing the dynamic field theory of spatial cognition across real and developmental time scales

“…A growing number of researchers have argued that we should take inspiration from the densely interconnected and dynamic nature of the brain to re-think cognition (e.g., Barsalou, 1999;Skarda & Freeman, 1987;. A centerpiece of this approach is to embrace the use of complex, dynamic neural networks to capture brain-behavior relations.…”

Using relations within conceptual systems to translate across conceptual systems

“…In the Barsalou's (1999) theory of perceptual symbol systems, concepts are not amodal, completely abstracted symbols, but rather are intrinsically perceptually based. He finds that detailed perceptual information is represented in concepts and that this information is used when reasoning about those concepts.…”