Invariant texture perception is harder with synthetic textures: Implications for models of texture processing

Abstract

Texture synthesis models have become a popular tool for studying the representations supporting texture processing in human vision. In particular, the summary statistics implemented in the Portilla-Simoncelli (PS) model support high-quality synthesis of natural textures, account for performance in crowding and search tasks, and may account for the response properties of V2 neurons. We chose to investigate whether or not these summary statistics are also sufficient to support texture discrimination in a task that required illumination invariance. Our observers performed a match-to-sample task using natural textures photographed with either diffuse overhead lighting or lighting from the side. Following a briefly presented sample texture, participants identified which of two test images depicted the same texture. In the illumination change condition, illumination differed between the sample and the matching test image. In the no change condition, sample textures and matching test images were identical. Critically, we generated synthetic versions of these images using the P-S model and also tested participants with these. If the statistics in the P-S model are sufficient for invariant texture perception, performance with synthetic images should not differ from performance in the original task. Instead, we found a significant cost of applying texture synthesis in both lighting conditions. We also observed this effect when power-spectra were matched across images (Experiment 2) and when sample and test images were drawn from unique locations in the parent textures to minimize the contribution of image-based processing (Experiment 3). Invariant texture processing thus depends upon measurements not implemented in the P-S algorithm.

Testing models of peripheral encoding using metamerism in an oddity paradigm

“…A. Wichmann, Braun, & Gegenfurtner, 2006). Because a potentially large set of features are available in natural scenes, and humans are very sensitive to many of them (Balas & Conlin, 2015;Emrith, Chantler, Green, Maloney, & Clarke, 2010;Gerhard, Wichmann, & Bethge, 2013;, demonstrating that human observers cannot discriminate modified (but model-matched) images from the original image is a strong test of the degree to which the transformations from retinal input to perception are captured by a candidate model. If images producing matching model responses also look the same, then any image properties not encoded by the model are perceptually unimportant.…”

A parametric texture model based on deep convolutional features closely matches texture appearance for humans

“…How, though, does it perform as a model of texture appearance in humans? Balas and colleagues (Balas, 2006(Balas, , 2008(Balas, , 2012Balas & Conlin, 2015) have reported a number of psychophysical investigations using the Portilla and Simoncelli (hereafter, PS) texture model that are relevant to this question. Balas (2006) quantified the relative importance of subsets of the PS statistics compared to the full set for matching the appearance of different classes of texture (periodic, structured, or asymmetric).…”

“…Finally, Balas and Conlin (2015) assessed whether the influence of illumination change on human texture perception could be captured by PS synthesis. Observers performed a match-to-sample task, in which they decided which of two match images depicted the same texture as a previously presented sample.…”

Neural Sensitivity to Natural Image Statistics Changes during Middle Childhood

“…Critically, though there are more recent models that make it possible to match synthetic textures to their parent images more closely with regard to these higher-order structural features, the absence of this matching using simpler models makes it possible to examine how adult visual processing is affected by the absence of these relationships from natural images. In general, removing these statistical regularities from natural images incurs a measurable cost in a range of tasks including invariant texture matching (Balas & Conlin, 2015a) and material categorization (Balas & Schmidt, 2017). Moreover, observers' ability to distinguish between putative "scene metamers" that are defined by matching parametric texture features like those described above (Freeman & Simoncelli, 2011;Wallis, Bethge & Wichmann, 2016) suggests that the adult visual system is sensitive to the absence of higher-order statistical regularities that are absent from synthetic images rendered via these models.…”