Modelling direct perceptual constraints on action selection: The Naming and Action Model (NAM)

Abstract

There is increasing experimental and neuropsychologica l evidence that action selection is directly constrained by perceptual information from objects as well as by more abstract semantic knowledge. To capture this evidence, we develop a new connectionist model of action and name selection from objects-NAM (Naming and Action Model), based on the idea that action selection is determined by convergent input from both visual structural descriptions and abstract semantic knowledge. We show that NAM is able to simulate evidence for a direct route to action selection from both normal subjects (Experiments 1 and 2) and neuropsychologica l patients (Experiments 3-6). The model provides a useful framework for understanding how perceptual knowledge influences action selection.

Direct and indirect effects of action on object classification

“…Nevertheless, the effects of action-associated information (handgrip and type of movement) tended to be consistent across decisions (affecting both yes and no responses) and stronger when action, rather than name, decisions were made to the objects (Experiments 1 and 2). This is in accordance with the predictions of a dualroute account of object processing, in which action-based representations are activated by both the structural and the semantic properties of objects (Riddoch et al, 1989;Yoon et al, 2002). According to this account, action decisions will be sensitive to the visual properties of episodes of action performed with objects, including handgrip and movement, because (1) these associated properties of action are represented in our stored knowledge of the actions performed with objects and (2) action knowledge is evoked directly by objects, without depending on prior access to more abstract semantic knowledge (e.g., knowledge about where objects might typically be found).…”

“…To account for this pattern of results, Riddoch et al (1989) proposed that damage to a direct visual route "blocked" an intact semantic route for action (evident when the patients gestured to object names; see Figure 1, Lesion Location C). Yoon, Heinke, and Humphreys (2002) have modeled these patterns of impairment in terms of a convergent route model of action. In this model, activation from separate semantic and visual routes converges to drive the retrieval of appropriate categories of actions for objects.…”

Vision, action and language unified through embodiment

“…Such a grounded and embodied approach to language design is consistent with the theories of the grounding of language discussed above. In these models there exists an intrinsic link between the communication symbols (words) used by the agent and its own cognitive representations (meanings) of the perceptual and sensorimotor interaction with the external world (referents) (Steels & Vogt, 1997;Steels, 2003;Yoon, Heinke & Humphreys, 2002). Cangelosi, Hourdakis, and Tikhanoff (2006) proposed a neural network model in a robotic set-up as a model of language acquisition.…”

A role for action knowledge in visual object identification

“…Dixon, Desmarais, Gojmerac, Schweizer, and Bub (2002) the authors to conclude that objects had privileged access to action knowledge compared to other forms of semantic knowledge, and that this advantage was contingent on learned associations between objects and actions. Yoon, Heinke, and Humphreys (2002) proposed a connectionist model of action selection and name selection, the naming and action model (NAM), which builds on the HIT model and explains how objects could have preferential access to actions via a direct route to action in addition to an indirect route via semantics. According to this model, and similar to the HIT model, the visual presentation of an object will activate the structural description system, first through input units that encode the visual attributes of the object presented, then through a network that captures the structural similarity between objects.…”