Repetitive presentation of the same visual stimulus entails a response decrease in the action potential discharge of neurons in various areas of the monkey visual cortex. It is still unclear whether this repetition suppression effect is also present in single neurons in cortical premotor areas responding to visual stimuli, as suggested by the human functional magnetic resonance imaging literature. Here we report the responses of 'mirror neurons' in monkey area F5 to the repeated presentation of action movies. We find that most single neurons and the population at large do not show a significant decrease of the firing rate. On the other hand, simultaneously recorded local field potentials exhibit repetition suppression. As local field potentials are believed to be better linked to the blood-oxygen-level-dependent (BOLD) signal exploited by functional magnetic resonance imaging, these findings suggest caution when trying to derive conclusions on the spiking activity of neurons in a given area based on the observation of BOLD repetition suppression.

The visual recognition of actions is an important visual function that is critical for motor learning and social communication. Actionselective neurons have been found in different cortical regions, including the superior temporal sulcus, parietal and premotor cortex. Among those are mirror neurons, which link visual and motor representations of body movements. While numerous theoretical models for the mirror neuron system have been proposed, the computational basis of the visual processing of goal-directed actions remains largely unclear. While most existing models focus on the possible role of motor representations in action recognition, we propose a model showing that many critical properties of action-selective visual neurons can be accounted for by well-established visual mechanisms. Our model accomplishes the recognition of hand actions from real video stimuli, exploiting exclusively mechanisms that can be implemented in a biologically plausible way by cortical neurons. We show that the model provides a unifying quantitatively consistent account of a variety of electrophysiological results from action-selective visual neurons. In addition, it makes a number of predictions, some of which could be confirmed in recent electrophysiological experiments.

Humans derive causality judgments reliably from highly abstract stimuli, such as moving discs that bump into each other [1]. This fascinating visual capability emerges gradually during human development [2], perhaps as consequence of sensorimotor experience [3]. Human functional imaging studies suggest an involvement of the "action observation network" in the processing of such stimuli [4, 5]. In addition, theoretical studies suggest a link between the computational mechanisms of action and causality perception [6, 7], consistent with the fact that both functions require an analysis of sequences of spatiotemporal relationships between interacting stimulus elements. Single-cell correlates of the perception of causality are completely unknown. In order to find such neural correlates, we investigated the responses of "mirror neurons" in macaque premotor area F5 [8, 9]. These neurons respond during the observation as well as during the execution of actions and show interesting invariances, e.g., with respect to the stimulus view [10], occlusions [11], or whether an action is really executed or suppressed [12]. We investigated the spatiotemporal properties of the visual responses of mirror neurons to naturalistic hand action stimuli and to abstract stimuli, which specified the same causal relationships. We found a high degree of generalization between these two stimulus classes. In addition, many features that strongly reduced the similarity of the response patterns coincided with the ones that also destroy the perception of causality in humans. This implies an overlap of neural structures involved in the processing of actions and the visual perception of causality at the single-cell level.

The efficient prediction of the behavior of others requires the recognition of their actions and an understanding of their action goals. In humans, this process is fast and extremely robust, as demonstrated by classical experiments showing that human observers reliably judge causal relationships and attribute interactive social behavior to strongly simplified stimuli consisting of simple moving geometrical shapes. While psychophysical experiments have identified critical visual features that determine the perception of causality and agency from such stimuli, the underlying detailed neural mechanisms remain largely unclear, and it is an open question why humans developed this advanced visual capability at all. We created pairs of naturalistic and abstract stimuli of hand actions that were exactly matched in terms of their motion parameters. We show that varying critical stimulus parameters for both stimulus types leads to very similar modulations of the perception of causality. However, the additional form information about the hand shape and its relationship with the object supports more fine-grained distinctions for the naturalistic stimuli. Moreover, we show that a physiologically plausible model for the recognition of goal-directed hand actions reproduces the observed dependencies of causality perception on critical stimulus parameters. These results support the hypothesis that selectivity for abstract action stimuli might emerge from the same neural mechanisms that underlie the visual processing of natural goal-directed action stimuli. Furthermore, the model proposes specific detailed neural circuits underlying this visual function, which can be evaluated in future experiments.

Computational models are fundamentally important for testing the feasibility of theories of the visual processing of body movements and for deriving well-defined theoretical predictions that can be tested experimentally. A computational model is proposed for the recognition of transitive and nontransitive hand actions from real videos that reproduces several key neurophysiological properties of the action perception system. Limitations of the proposed model, along with novel predictions and areas for future research, are discussed.

Abstract. The recognition of transitive, goal-directed actions requires a sensible balance between the representation of specific shape details of effector and goal object and robustness with respect to image transformations. We present a biologically-inspired architecture for the recognition of transitive actions from video sequences that integrates an appearancebased recognition approach with a simple neural mechanism for the representation of the effector-object relationship. A large degree of position invariance is obtained by nonlinear pooling in combination with an explicit representation of the relative positions of object and effector using neural population codes. The approach was tested on real videos, demonstrating successful invariant recognition of grip types on unsegmented video sequences. In addition, the algorithm reproduces and predicts the behavior action-selective neurons in parietal and prefrontal cortex.

No abstract

No abstract