An image-computable model of human visual shape similarity

Abstract

Shape is a defining feature of objects. Yet, no image-computable model accurately predicts how similar or different shapes appear to human observers. To address this, we developed a model ('ShapeComp'), based on over 100 shape features (e.g., area, compactness, Fourier descriptors). When trained to capture the variance in a database of >25,000 animal silhouettes, ShapeComp predicts human shape similarity judgments almost perfectly (r 2 >0.99) without fitting any parameters to human data. To test the model, we created carefully selected arrays of complex novel shapes using a Generative Adversarial Network trained on the animal silhouettes, which we presented to observers in a wide range of tasks. Our findings show that human shape perception is inherently multidimensional and optimized for comparing natural shapes. ShapeComp outperforms conventional metrics, and can also be used to generate perceptually uniform stimulus sets, making it a powerful tool for investigating shape and object representations in the human brain.

Contact lenses, the reverse Pulfrich effect, and anti-Pulfrich monovision corrections

“…defocus blur, binocular disparity, and motion) provide reason for optimism 8,[56][57][58] . Models that compute cue values directly from images ('image-computable models') have found recent success in predicting human performance in a range of visual tasks [59][60][61][62][63][64][65][66] . However, to our knowledge, there exists no theoretical or empirical work that tightly links the properties of natural images to processing speed.…”

Generative adversarial networks unlock new methods for cognitive science

“…Using a well-trained face GAN [9] (an option also explored in [50]), future work could complement important studies like [51], which relied on hand-picked features and manual photo editing. Another example is a study [52] investigating how we represent shapes, which is crucial for object recognition and many other tasks. The authors of this study present ShapeComp [52], a space with 100 handpicked features, and even trained a GAN to generate new shapes for validation purposes.…”

“…Another example is a study [52] investigating how we represent shapes, which is crucial for object recognition and many other tasks. The authors of this study present ShapeComp [52], a space with 100 handpicked features, and even trained a GAN to generate new shapes for validation purposes. Future work, however, could study the qualities of their GAN's latent space as a shape space in its own right.…”

Contact lenses can cause the reverse Pulfrich effect and anti-Pulfrich monovision corrections can eliminate it

“…defocus blur, binocular disparity, and motion) provide reason for optimism (Burge & Geisler, 2011;2012;2015). Models that compute cue values directly from images ('image-computable models') have found recent success in predicting human performance (Chin & Burge, 2020;Kane, Bex, & Dakin, 2011;Morgenstern et al, 2020;Schütt & Wichmann, 2017;Sebastian, Abrams, & Geisler, 2017). However, to our knowledge, there exists no theoretical or empirical work that tightly links the properties of natural images to processing speed.…”