An applet for the Gabor similarity scaling of the differences between complex stimuli

Abstract

It is widely accepted that after the first cortical visual area, V1, a series of stages achieves a representation of complex shapes, such as faces and objects, so that they can be understood and recognized. A major challenge for the study of complex shape perception has been the lack of a principled basis for scaling of the physical differences between stimuli so that their similarity can be specified, unconfounded by early-stage differences. Without the specification of such similarities, it is difficult to make sound inferences about the contributions of later stages to neural activity or psychophysical performance. A Web-based app is described that is based on the Malsburg Gabor-jet model (Lades et al., 1993), which allows easy specification of the V1 similarity of pairs of stimuli, no matter how intricate. The model predicts the psycho physical discriminability of metrically varying faces and complex blobs almost perfectly (Yue, Biederman, Mangini, von der Malsburg, & Amir, 2012), and serves as the input stage of a large family of contemporary neurocomputational models of vision.Keywords 2-D shape and form . Similarity . Face perception Consider the problem of determining whether people or monkeys are more sensitive to differences in nonaccidental properties (NAPs)-whether a contour is straight or curved, for example-than to differences in metric properties (MPs)-such as differences in degrees of curvature. If we assume that the sensitivity to differences in NAPs arises at a stage in the ventral pathway later than V1, how can the physical properties of the stimuli be selected in a principled manner, so that the comparisons are not confounded with differences in V1 activation? The same methodological problem arises if an investigator wishes to determine whether observers are more sensitive to differences in facial expression than to differences in identity (or sex, or orientation in depth, etc.). This problem arises not only in psychophysical scaling of stimuli, but also with studies designed to more directly reflect the underlying neural correlates, such as fMRI fastadaptation designs and single-unit recordings. It can be argued that this problem of the scaling of shape similarity had been a major reason why, despite shape being the major input into visual cognition, the rigorous study of shape perception had clearly lagged the study of other perceptual attributes, such as color, motion, or stereo. The value of an intuitive implementation of the Gabor-jet modelDespite the utility in employing such a scaling system, the Gaborjet model is somewhat mathematically dense and cumbersome to explain to the uninitiated, thus diminishing its accessibility. Here, we introduce a Web-based applet designed to provide an engaging, graphically oriented guided tour of the model. The applet allows users to upload their own images, observe the transformations and computations made by the algorithm, customize the visualization of different processes, and retrieve a ranking of dissimilarity values for pairs of images. S...

Seeing structure: Shape skeletons modulate perceived similarity

“…Finally, in addition to these shape features, we also computed an independent measure of shape similarity using the Malsburg Gabor-jet model (Lades et al, 1993;Margalit, Biederman, Herald, Yue, & von der Malsburg, 2016), which has been shown to robustly track human discrimination performance for metric differences between shapes (Yue, Biederman, Mangini, von der Malsburg, & Amir, 2012). Inspired by the Gabor-like filtering of simple cells in V1 (Jones & Palmer, 1987), this model overlays sets (or "jets") of 40 Gabor filters (5 scales × 8 orientations) on each pixel of a 128 × 128-pixel image and calculates the convolution of the input image with each filter, storing both the magnitude and the phase of the filtered image.…”

“…Average psychophysical distances according to the Gabor-jet model(Lades et al, 1993;Margalit et al, 2016) …”

How Configural Is the Configural Superiority Effect? A Neuroimaging Investigation of Emergent Features in Visual Cortex

“…This may be accounted for by the fact that while the scrambled and experimental stimuli were originally designed to have identical physical size (in terms of pixel count; for examples of stimuli, see Figure 1 ), it might be that the density and distribution of these pixels differ across the different types of stimuli, leading to this unexpected effect. This interpretation was supported by an image analysis using V1-like Gabor jet-filter model (Yue et al, 2012; Margalit et al, 2016, see Supplementary Information).…”

Skeletal descriptions of shape provide unique perceptual information for object recognition

“…Gabor-jet (GBJ) model. The GBJ model is a low-level model of image similarity inspired by the response profile of complex cells in early visual cortex 46 . It has been shown to scale with human psychophysical dissimilarity judgments of faces and simple objects 77 .…”

“…In all cases, we examined the unique contributions of skeletal structures in object recognition by contrasting the shape skeleton with models of vision that do not explicitly incorporate a skeletal structure, but are nevertheless predictive of human object recognition. These models included those that describe visual similarity by their image statistics, namely, the Gabor-Jet (GBJ) model 46 and GIST model 47 , as well as biologically plausible neural networks models, namely, the HMAX model 40 and AlexNet, a CNN pre-trained to identify objects 41 . To anticipate our findings, a model of skeletal similarity was predictive of participants' perceptual similarity and classification judgments even when accounting for these other models, suggesting that skeletal descriptions of shape play a crucial role in human object recognition, independent of other models of shape and object perception.…”