Interactive Activation and Mutual Constraint Satisfaction in Perception and Cognition

Abstract

In a seminal 1977 article, Rumelhart argued that perception required the simultaneous use of multiple sources of information, allowing perceivers to optimally interpret sensory information at many levels of representation in real time as information arrives. Building on Rumelhart's arguments , we present the Interactive Activation hypothesis-the idea that the mechanism used in perception and comprehension to achieve these feats exploits an interactive activation process implemented through the bidirectional propagation of activation among simple processing units. We then examine the interactive activation model of letter and word perception and the TRACE model of speech perception, as early attempts to explore this hypothesis, and review the experimental evidence relevant to their assumptions and predictions. We consider how well these models address the computational challenge posed by the problem of perception, and we consider how consistent they are with evidence from behavioral experiments. We examine empirical and theoretical controversies surrounding the idea of interactive processing, including a controversy that swirls around the relationship between interactive computation and optimal Bayesian inference. Some of the implementation details of early versions of interactive activation models caused deviation from optimality and from aspects of human performance data. More recent versions of these models, however, overcome these deficiencies. Among these is a model called the multinomial interactive activation model, which explicitly links interactive activation and Bayesian computations. We also review evidence from neurophysiological and neuroimaging studies supporting the view that interactive processing is a characteristic of the perceptual processing machinery in the brain. In sum, we argue that a computational analysis, as well as behavioral and neuroscience evidence , all support the Interactive Activation hypothesis. The evidence suggests that contemporary Correspondence should be sent to versions of models based on the idea of interactive activation continue to provide a basis for efforts to achieve a fuller understanding of the process of perception.

Flexible and adaptive processes in speech perception

“…However, interactions between semantic information and intelligibility observed in previous studies, in parietal (Golestani et al, 2013), and in frontal and temporal areas (Davis et al, 2011;Gow et al, 2008;Guediche, Salvata & Blumstein, 2013;Obleser & Kotz, 2010) suggest the possibility that at least some properties of the details of the acoustic speech signal are retained throughout different levels of processing. The results are consistent with highly interactive models such as those that support mutual constraint satisfaction (e. g., McClelland et al, 2014).…”

The multimodal nature of spoken word processing in the visual world: Testing the predictions of alternative models of multimodal integration

“…In this paper we implement each of these alternative architectures in cognitively plausible (McClelland, Mirman, Bolger, & Khaitan, 2014) computational models. In both cases spoken word recognition and spoken word comprehension are framed in terms of multimodal constraint satisfaction (cf.…”

“…In both cases spoken word recognition and spoken word comprehension are framed in terms of multimodal constraint satisfaction (cf. MacDonald, Pearlmutter, & Seidenberg, 1994;McClelland, Rumelhart, & Hinton, 1986;McClelland et al, 2014), with words conceived as entities that connect representations across multiple modalities (e.g., phonological, orthographic, semantic, visual, etc.). In both models, speech processing occurs in a multimodal context, with activation of information passing between modalities to reflect real time sensory input.…”

The Visual System's Internal Model of the World

“…The key distinction of this class of models with the first class is in its emphasis on perception being an interactive process, involving top-down feedback. The model is based on the association principle and can be tied to constraint satisfaction and probabilistic models of perception [15], [16]. O’ Reilly et al’s [17] Leabra cognitive architecture may be the most powerful instantiation of this class of models for recurrent processing in object recognition.…”