It is well established that movement planning recruits motor-related cortical brain areas in preparation for the forthcoming action. Given that an integral component to the control of action is the processing of sensory information throughout movement, we predicted that movement planning might also modulate early sensory cortical areas, readying them for sensory processing during the unfolding action. To test this hypothesis, we performed 2 human functional magnetic resonance imaging studies involving separate delayed movement tasks and focused on premovement neural activity in early auditory cortex, given the area’s direct connections to the motor system and evidence that it is modulated by motor cortex during movement in rodents. We show that effector-specific information (i.e., movements of the left vs. right hand in Experiment 1 and movements of the hand vs. eye in Experiment 2) can be decoded, well before movement, from neural activity in early auditory cortex. We find that this motor-related information is encoded in a separate subregion of auditory cortex than sensory-related information and is present even when movements are cued visually instead of auditorily. These findings suggest that action planning, in addition to preparing the motor system for movement, involves selectively modulating primary sensory areas based on the intended action.

The individual shape of the human body, including the geometry of its articulated structure and the distribution of weight over that structure, influences the kinematics of a person's movements. How sensitive is the visual system to inconsistencies between shape and motion introduced by retargeting motion from one person onto the shape of another? We used optical motion capture to record five pairs of male performers with large differences in body weight, while they pushed, lifted, and threw objects. From these data, we estimated both the kinematics of the actions as well as the performer's individual body shape. To obtain consistent and inconsistent stimuli, we created animated avatars by combining the shape and motion estimates from either a single performer or from different performers. Using these stimuli we conducted three experiments in an immersive virtual reality environment. First, a group of participants detected which of two stimuli was inconsistent. Performance was very low, and results were only marginally significant. Next, a second group of participants rated perceived attractiveness, eeriness, and humanness of consistent and inconsistent stimuli, but these judgements of animation characteristics were not affected by consistency of the stimuli. Finally, a third group of participants rated properties of the objects rather than of the performers. Here, we found strong influences of shape-motion inconsistency on perceived weight and thrown distance of objects. This suggests that the visual system relies on its knowledge of shape and motion and that these components are assimilated into an altered perception of the action outcome. We propose that the visual system attempts to resist inconsistent interpretations of human animations. Actions involving object manipulations present an opportunity for the visual system to reinterpret the introduced inconsistencies as a change in the dynamics of an object rather than as an unexpected combination of body shape and body motion.

It is well established that movement planning recruits motor-related cortical brain areas in preparation for the forthcoming action. Given that an integral component of the control of action is the processing of sensory information throughout movement, we predicted that movement planning also involves preparing early sensory cortical areas for participation in the impending behaviour. To test this hypothesis, we had human participants perform an object manipulation task wherein we focused on activity in early human auditory cortex, given the role of auditory signals in the sensorimotor control of such tasks and because of its known ipsilateral connections with the motor system.Here we show, using functional MRI and pattern classification methods, that information related to the limb to be used to grasp and lift an object can be decoded, well before movement, from neural activity patterns in early auditory cortex. We further show that the decoding of this motor-related information occurs in a separate subregion of auditory cortex than the decoding of the auditory sensory information used to instruct, and prompt preparation of, the hand actions. Together, this evidence suggests that action planning, in addition to preparing the motor system for movement, involves the task-specific preparation of primary sensory areas, such that they are set up to appropriately process sensory information arising during the unfolding movement.

No abstract

Perception of human action depends on both the body shape and motion of a performer. We can indirectly perceive the properties of an object being acted upon even when visual information is limited and the object itself is not visible; we accomplish this using internal models of a body’s dynamics and an action’s kinematics (Runeson & Frykholm, 1981). We are also sensitive to correlations between a performer’s shape and motion, known as internal consistency (Runeson & Frykholm, 1983). To investigate how decorrelating shape and motion affects indirect object perception, we ran an experiment where participants watched realistic avatars of performers manipulating invisible objects. Unbeknownst to participants, half of the stimuli were internally inconsistent: the shape of one performer was combined with the motion of a performer with a dissimilar body shape. Participants saw sled pushes, beanbag throws, and box lifts, and estimated the sled weight, throw distance, or box weight. For sled pushes, there was a shape-motion interaction such that heavy bodies were perceived as pushing heavier weights when animated with motion from light performers, and light bodies were perceived as pushing lighter weights when animated with motion from heavy performers. In contrast, participants estimated beanbag throw distance primarily from performer motion. Interpretation of the box lift data is more complex. In conclusion, the way in which our visual system combines shape and motion information depends on the role of body shape and centre of mass on the outcome of an action.

The individual shape of the human body, including the geometry of its articulated structure and the distribution of weight over that structure, influences the kinematics of a person's movements. How sensitive is the visual system to inconsistencies between shape and motion introduced by retargeting motion from one person onto the shape of another? We used optical motion capture to record five pairs of male performers with large differences in body weight, while they pushed, lifted, and threw objects. Based on a set of 67 markers, we estimated both the kinematics of the actions as well as the performer's individual body shape. To obtain consistent and inconsistent stimuli, we created animated avatars by combining the shape and motion estimates from either a single performer or from different performers. In a virtual reality environment, observers rated the perceived weight or thrown distance of the objects. They were also asked to explicitly discriminate between consistent and hybrid stimuli. Observers were unable to accomplish the latter, but hybridization of shape and motion influenced their judgements of action outcome in systematic ways. Inconsistencies between shape and motion were assimilated into an altered perception of the action outcome.