One of the more stunning examples of the resourcefulness of human vision is the ability to see 'biological motion', which was first shown with an adaptation of earlier cinematic work: illumination of only the joints of a walking person is enough to convey a vivid, compelling impression of human animation, although the percept collapses to a jumble of meaningless lights when the walker stands still. The information is sufficient to discriminate the sex and other details of the walker, and can be interpreted by young infants. Here we measure the ability of the visual system to integrate this type of motion information over space and time, and compare this capacity with that for viewing simple translational motion. Sensitivity to biological motion increases rapidly with the number of illuminated joints, far more rapidly than for simple motion. Furthermore, this information is summed over extended temporal intervals of up to 3 seconds (eight times longer than for simple motion). The steepness of the summation curves indicates that the mechanisms that analyse biological motion do not integrate linearly over space and time with constant efficiency, as may occur for other forms of complex motion, but instead adapt to the nature of the stimulus.

Stereoscopic vision relies mainly on relative depth differences between objects rather than on their absolute distance in depth from where the eyes fixate. However, relative disparities are computed from absolute disparities, and it is not known where these two stages are represented in the human brain. Using functional MRI (fMRI), we assessed absolute and relative disparity selectivity with stereoscopic stimuli consisting of pairs of transparent planes in depth in which the absolute and relative disparity signals could be independently manipulated (at a local spatial scale). In experiment 1, relative disparity was kept constant, while absolute disparity was varied in one-half the blocks of trials ("mixed" blocks) and kept constant in the remaining one-half ("same" blocks), alternating between blocks. Because neuronal responses undergo adaptation and reduce their firing rate following repeated presentation of an effective stimulus, the fMRI signal reflecting activity of units selective for absolute disparity is expected to be smaller during "same" blocks as compared with "mixed" ones. Experiment 2 similarly manipulated relative disparity rather than absolute disparity. The results from both experiments were consistent with adaptation with differential effects across visual areas such that 1) dorsal areas (V3A, MT+/V5, V7) showed more adaptation to absolute than to relative disparity; 2) ventral areas (hV4, V8/V4alpha) showed an equal adaptation to both; and 3) early visual areas (V1, V2, V3) showed a small effect in both experiments. These results indicate that processing in dorsal areas may rely mostly on information about absolute disparities, while ventral areas split neural resources between the two types of stereoscopic information so as to maintain an important representation of relative disparity.

The ability to interpret and predict other people's actions is highly evolved in humans and is believed to play a central role in their cognitive behavior. However, there is no direct evidence that this ability confers a tangible benefit to sensory processing. Our quantitative behavioral experiments show that visual discrimination of a human agent is influenced by the presence of a second agent. This effect depended on whether the two agents interacted (by fighting or dancing) in a meaningful synchronized fashion that allowed the actions of one agent to serve as predictors for the expected actions of the other agent, even though synchronization was irrelevant to the visual discrimination task. Our results demonstrate that action understanding has a pervasive impact on the human ability to extract visual information from the actions of other humans, providing quantitative evidence of its significance for sensory performance.

This brief review article brings together a series of related experiments in psychophysics and physiology that show striking similarities between measurements in human observers and in single neurons. We consider seven pairs of primary research articles, each pair consisting of one paper in physiology and one in psychophysics, and we highlight common features between receptive and perceptive fields obtained using reverse correlation. We conclude by discussing how to assess the validity of perceptive fields as predictors of human responses, and by deriving a novel expression for the maximum trial-by-trial predictability attainable by any model for any psychophysical task.

No abstract

Our visual system constantly selects salient features in the environment, so that only those features are attended and targeted by further processing efforts to identify them. Models of feature detection hypothesize that salient features are localized based on contrast energy (local variance in intensity) in the visual stimulus. This hypothesis, however, has not been tested directly. We used psychophysical reverse correlation to study how humans detect and identify basic image features (bars and short line segments). Subjects detected a briefly flashed 'target bar' that was embedded in 'noise bars' that randomly changed in intensity over space and time. By studying how the intensity of the noise bars affected performance, we were able to dissociate two processing stages: an early 'detection' stage, whereby only locations of high-contrast energy in the image are selected, followed (after approximately 100 ms) by an 'identification' stage, whereby image intensity at selected locations is used to determine the identity (whether bright or dark) of the target.

Human sensory processing can be viewed as a functional H mapping a stimulus vector s into a decisional variable r. We currently have no direct access to r; rather, the human makes a decision based on r in order to drive subsequent behavior. It is this (typically binary) decision that we can measure. For example, there may be two external stimuli s([0]) and s([1]), mapped onto r([0]) and r([1]) by the sensory apparatus H; the human chooses the stimulus associated with largest r. This kind of decisional transduction poses a major challenge for an accurate characterization of H. In this article, we explore a specific approach based on a behavioral variant of reverse correlation techniques, where the input s contains a target signal corrupted by a controlled noisy perturbation. The presence of the target signal poses an additional challenge because it distorts the otherwise unbiased nature of the noise source. We consider issues arising from both the decisional transducer and the target signal, their impact on system identification, and ways to handle them effectively for system characterizations that extend to second-order functional approximations with associated small-scale cascade models.

Our two eyes obtain slightly different views of the world. The resulting differences in the two retinal images, called binocular disparities, provide us with a stereoscopic sense of depth. The primary visual cortex (V1) contains neurons that are selective for the disparity of individual elements in an image, but this information must be further analysed to complete the stereoscopic process. Here we apply the psychophysical technique of reverse correlation to investigate disparity processing in human vision. Observers viewed binocular random-dot patterns, with 'signal' dots in a specific depth plane plus 'noise' dots with randomly assigned disparities. By examining the correlation between the observers' ability to detect the plane and the particular sample of 'noise' disparities presented on each trial, we revealed detection 'filters', whose disparity selectivity was remarkably similar to that of individual neurons in monkey V1. Moreover, if the noise dots were of opposite contrast in the two eyes, the tuning inverted, just like the response patterns of V1 neurons. Reverse correlation appears to probe disparity processing at the earliest stages of binocular combination, prior to the generation of a full stereoscopic depth percept.