A summary-statistic representation in peripheral vision explains visual crowding

Abstract

Peripheral vision provides a less faithful representation of the visual input than foveal vision. Nonetheless, we can gain a lot of information about the world from our peripheral vision, for example in order to plan eye movements. The phenomenon of crowding shows that the reduction of information available in the periphery is not merely the result of reduced resolution. Crowding refers to visual phenomena in which identification of a target stimulus is significantly impaired by the presence of nearby stimuli, or flankers. What information is available in the periphery? We propose that the visual system locally represents peripheral stimuli by the joint statistics of responses of cells sensitive to different position, phase, orientation, and scale. This “textural” representation by summary statistics predicts the subjective “jumble” of features often associated with crowding. We show that the difficulty of performing an identification task within a single pooling region using this representation of the stimuli is correlated with peripheral identification performance under conditions of crowding. Furthermore, for a simple stimulus with no flankers, this representation can be adequate to specify the stimulus with some position invariance. This provides evidence that a unified neuronal mechanism may underlie peripheral vision, ordinary pattern recognition in central vision, and texture perception. A key component of our methodology involves creating visualizations of the information available in the summary statistics of a stimulus. We call these visualizations “mongrels” and show that they are highly useful in examining how the early visual system represents the visual input. Mongrels enable one to study the “equivalence classes” of our model, i.e., the sets of stimuli that map to the same representation according to the model.

Image statistics underlying natural texture selectivity of neurons in macaque V4

“…Considering that curvature can also be regarded as a higher-order combinatory feature of Gabor filter output, it may be that there is shared tuning between these contour fragments and textures. However, texture synthesis does not capture global image structures and cannot reproduce inhomogeneous image features such as object contours (20), although it may explain visual discriminability of those features in peripheral visual field (21,41,42). It is therefore also plausible that tuning to geometric shapes or curvatures is hidden in the unexplained variances in the neural responses recorded in this study, or that they can be attributed to a different population of V4 neurons tuned to those features.…”

The effects of sampling and internal noise on the representation of ensemble average size

“…In contrast, work from the ensemble feature literature has often taken a sample to be one of the segmentable objects within the ensemble and has implemented sampling without replacement (e.g., Haberman & Whitney, 2010;. We believe that extracting ensemble features may not rely on segmenting individual items, but rather may rely on mechanisms more similar to texture processing-a suggestion that has also been made elsewhere (Balas, Nakano, & Rosenholtz, 2009;Dakin, Tibber, Greenwood, Kingdom, & Morgan, 2011;Freeman & Simoncelli, 2011;Haberman & Whitney, 2010;Parkes et al, 2001). …”

Summary statistics in auditory perception

“…The averaging that has been characterized in vision occurs over spatial regions of the visual field rather than time and is most evident in the periphery, where the dependence on ensemble statistics has been related to the phenomenon of crowding [42][43][44][45] . Statistical measures also seem to dominate visual perception in the absence of attention 46 .…”