Event-related potentials (ERPs) associated with face perception were recorded with scalp electrodes from normal volunteers. Subjects performed a visual target detection task in which they mentally counted the number of occurrences of pictorial stimuli from a designated category such us butterflies. In separate experiments, target stimuli were embedded within a series of other stimuli including unfamiliar human faces and isolated face components, inverted faces, distorted faces, animal faces, and other nonface stimuli. Unman faces evoked a negative potential at 172 msec (N170), which was absent from the ERPs elicited by other animate and inanimate nonface stimuli. N170 was largest over the posterior temporal scalp and was larger over the right than the left hemisphere. N170 was delayed when faces were presented upside-down, but its amplitude did not change. When presented in isolation, eyes elicited an N170 that was significantly larger than that elicited by whole faces, while noses and lips elicited small negative ERPs about 50 msec later than N170. Distorted human faces, in which the locations of inner face components were altered, elicited an N170 similar in amplitude to that elicited by normal faces. However, faces of animals, human hands, cars, and items of furniture did not evoke N170. N170 may reflect the operation of a neural mechanism tuned to detect (as opposed to identify) human faces, similar to the "structural encoder" suggested by Bruce and Young (1986). A similar function has been proposed for the face-selective N200 ERP recorded from the middle fusiform and posterior inferior temporal gyri using subdural electrodes in humans (Allison, McCarthy, Nobre, Puce, & Belger, 1994c). However, the differential sensitivity of N170 to eyes in isolation suggests that N170 may reflect the activation of an eye-sensitive region of cortex. The voltage distribution of N170 over the scalp is consistent with a neural generator located in the occipitotemporal sulcus lateral to the fusiform/inferior temporal region that generates N200.

No abstract

We sought to determine whether regions of extrastriate visual cortex could be activated in subjects viewing eye and mouth movements that occurred within a stationary face. Eleven subjects participated in three to five functional magnetic resonance imaging sessions in which they viewed moving eyes, moving mouths, or movements of check patterns that occurred in the same spatial location as the eyes or mouth. In each task, the stimuli were superimposed on a radial background pattern that continually moved inward to control for the effect of movement per se. Activation evoked by the radial background was assessed in a separate control task. Moving eyes and mouths activated a bilateral region centered in the posterior superior temporal sulcus (STS). The moving check patterns did not appreciably activate the STS or surrounding regions. The activation by moving eyes and mouths was distinct from that elicited by the moving radial background, which primarily activated the posterior-temporal-occipital fossa and the lateral occipital sulcus-a region corresponding to area MT/V5. Area MT/V5 was also strongly activated by moving eyes and to a lesser extent by other moving stimuli. These results suggest that a superior temporal region centered in the STS is preferentially involved in the perception of gaze direction and mouth movements. This region of the STS may be functionally related to nearby superior temporal regions thought to be involved in lip-reading and in the perception of hand and body movement.

This and the following two papers describe event-related potentials (ERPs) evoked by visual stimuli in 98 patients in whom electrodes were placed directly upon the cortical surface to monitor medically intractable seizures. Patients viewed pictures of faces, scrambled faces, letter-strings, number-strings, and animate and inanimate objects. This paper describes ERPs generated in striate and peristriate cortex, evoked by faces, and evoked by sinusoidal gratings, objects and letter-strings. Short-latency ERPs generated in striate and peristriate cortex were sensitive to elementary stimulus features such as luminance. Three types of face-specific ERPs were found: (i) a surface-negative potential with a peak latency of approximately 200 ms (N200) recorded from ventral occipitotemporal cortex, (ii) a lateral surface N200 recorded primarily from the middle temporal gyrus, and (iii) a late positive potential (P350) recorded from posterior ventral occipitotemporal, posterior lateral temporal and anterior ventral temporal cortex. Face-specific N200s were preceded by P150 and followed by P290 and N700 ERPs. N200 reflects initial face-specific processing, while P290, N700 and P350 reflect later face processing at or near N200 sites and in anterior ventral temporal cortex. Face-specific N200 amplitude was not significantly different in males and females, in the normal and abnormal hemisphere, or in the right and left hemisphere. However, cortical patches generating ventral face-specific N200s were larger in the right hemisphere. Other cortical patches in the same region of extrastriate cortex generated grating-sensitive N180s and object-specific or letter-string-specific N200s, suggesting that the human ventral object recognition system is segregated into functionally discrete regions.

The perception of faces is sometimes regarded as a specialized task involving discrete brain regions. In an attempt to identi$ face-specific cortex, we used functional magnetic resonance imaging (fMRI) to measure activation evoked by faces presented in a continuously changing montage of common objects or in a similar montage of nonobjects. Bilateral regions of the posterior fusiform gyrus were activated by faces viewed among nonobjects, but when viewed among objects, faces activated only a focal right fusiform region. To determine whether this focal activation would occur for another category of familiar stimuli, subjects viewed flowers presented among nonobjects and objects. While flowers among nonobjects evoked bilateral fusiform activation, flowers among objects evoked no activation. These results demonstrate that both faces and flowers activate large and partially overlapping regions of inferior extrastriate cortex. A smaller region, located primarily in the right lateral fusiform gyrus, is activated specifically by faces.

Twelve normal subjects viewed alternating sequences of unfamiliar faces, unpronounceable nonword letterstrings, and textures while echoplanar functional magnetic resonance images were acquired in seven slices extending from the posterior margin of the splenium to near the occipital pole. These stimuli were chosen to elicit initial category-specific processing in extrastriate cortex while minimizing semantic processing. Overall, faces evoked more activation than did letterstrings. Comparing hemispheres, faces evoked greater activation in the right than the left hemisphere, whereas letterstrings evoked greater activation in the left than the right hemisphere. Faces primarily activated the fusiform gyrus bilaterally, and also activated the right occipitotemporal and inferior occipital sulci and a region of lateral cortex centered in the middle temporal gyrus. Letterstrings primarily activated the left occipitotemporal and inferior occipital sulci. Textures primarily activated portions of the collateral sulcus. In the left hemisphere, 9 of the 12 subjects showed a characteristic pattern in which faces activated a discrete region of the lateral fusiform gyrus, whereas letterstrings activated a nearby region of cortex within the occipitotemporal and inferior occipital sulci. These results suggest that different regions of ventral extrastriate cortex are specialized for processing the perceptual features of faces and letterstrings, and that these regions are intermediate between earlier processing in striate and peristriate cortex, and later lexical, semantic, and associative processing in downstream cortical regions.

The movements of the faces and bodies of other conspecifics provide stimuli of considerable interest to the social primate. Studies of single cells, field potential recordings and functional neuroimaging data indicate that specialized visual mechanisms exist in the superior temporal sulcus (STS) of both human and non-human primates that produce selective neural responses to moving natural images of faces and bodies. STS mechanisms also process simplified displays of biological motion involving point lights marking the limb articulations of animate bodies and geometrical shapes whose motion simulates purposeful behaviour. Facial movements such as deviations in eye gaze, important for gauging an individual's social attention, and mouth movements, indicative of potential utterances, generate particularly robust neural responses that differentiate between movement types. Collectively such visual processing can enable the decoding of complex social signals and through its outputs to limbic, frontal and parietal systems the STS may play a part in enabling appropriate affective responses and social behaviour.

It is widely recognized that viewing a speaker's face enhances vocal communication, although the neural substrates of this phenomenon remain unknown. We propose that the enhancement effect uses the ongoing oscillatory activity of local neuronal ensembles in the primary auditory cortex. Neuronal oscillations reflect rhythmic shifting of neuronal ensembles between high and low excitability states. Our hypothesis holds that oscillations are 'predictively' modulated by visual input, so that related auditory input arrives during a high excitability phase and is thus amplified. We discuss the anatomical substrates and key timing parameters that enable and constrain this effect. Our hypothesis makes testable predictions for future studies and emphasizes the idea that 'background' oscillatory activity is instrumental to cortical sensory processing. Seeing voicesOver 50 years ago, Sumby and Pollack [1] noted that viewing the face of a speaker increases the intelligibility of vocal communication. Despite the obvious power and ubiquity of this phenomenon, its specific neural underpinnings remain elusive. We hypothesize that visual amplification of speech is operating as early as the first stage of cortical auditory processing in Area A1 (see Glossary), and that the underlying process entails an elegant and efficient modulation or 'shaping' of ongoing neuronal oscillations. This effect could be a model for cortical modulatory processes in general. Audiovisual integration in the primary auditory cortex?In the past decade, functional magnetic resonance imaging (fMRI) studies have consistently pointed to the superior temporal sulcus (STS) region as a crucial part of the brain mechanism for audiovisual integration in speech perception [2]. The idea that the human STS is a key substrate for audiovisual speech processing makes sense, as it seems to correspond, at least in part, to the superior temporal polysensory (STP) cortex in the macaque monkey, a classical site of audiovisual convergence [3]. However, althoughCorresponding author: Schroeder, C.E. (schrod@nki.rfmh.org). HHS Public Access Author Manuscript Author ManuscriptAuthor ManuscriptAuthor Manuscript multisensory integration involves the STS, it does not seem to begin there, as shown by the rapidly mounting evidence implicating areas in and near the primary auditory cortex in audiovisual integration during speech processing [4][5][6]. Furthermore, although a precise physiological interpretation of these findings is limited by the indirect nature of noninvasive imaging measures, the recent increase in new findings on the multisensory properties of the auditory cortex in nonhuman primates (reviewed in Refs [7,8]) suggests that answers to some of the key mechanistic questions could be forthcoming. Given the parallel between human and monkey communication processing (Box 1), certain findings in monkeys are directly relevant to understanding the brain mechanisms of audiovisual communication in humans. HypothesisThe traditional understanding of the neural underpinnin...