The spatiotemporal neural dynamics underlying perceived similarity for real-world objects

Cichy, Radoslaw Martin; Kriegeskorte, Nikolaus; Jozwik, Kamila M.; Bosch, Jasper J.F. van den; Charest, Ian

doi:10.1016/j.neuroimage.2019.03.031

Cited by 66 publications

(77 citation statements)

References 101 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…One important difference between our experiment and previous studies that have used inverse MDS is that rather than selecting items that fall into clear categorical groups such as faces, bodies or animals 26,28,30 , the items in our set were highly heterogeneous. Our stimuli and design therefore permit a unique exploration of the characteristics that observers use to define and differentiate between objects in the absence of cues that would otherwise bias sorting criteria.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Object responses are highly malleable, rather than invariant, with changes in object appearance

Holler

Fabbri

Snow

2020

Sci Rep

View full text Add to dashboard Cite

theoretical frameworks of human vision argue that object responses remain stable, or 'invariant', despite changes in viewing conditions that can alter object appearance but not identity. Here, in a major departure from previous approaches that have relied on two-dimensional (2-D) images to study object processing, we demonstrate that changes in an object's appearance, but not its identity, can lead to striking shifts in behavioral responses to objects. We used inverse multidimensional scaling (MDS) to measure the extent to which arrangements of objects in a sorting task were similar or different when the stimuli were displayed as scaled 2-D images, three-dimensional (3-D) augmented reality (AR) projections, or real-world solids. We were especially interested in whether sorting behavior in each display format was based on conceptual (e.g., typical location) versus physical object characteristics. We found that 2-D images of objects were arranged according to conceptual (typical location), but not physical, properties. AR projections, conversely, were arranged primarily according to physical properties such as real-world size, elongation and weight, but not conceptual properties. Real-world solid objects, unlike both 2-D and 3-D images, were arranged using multidimensional criteria that incorporated both conceptual and physical object characteristics. our results suggest that object responses can be strikingly malleable, rather than invariant, with changes in the visual characteristics of the stimulus. The findings raise important questions about limits of invariance in object processing, and underscore the importance of studying responses to richer stimuli that more closely resemble those we encounter in real-world environments.Understanding how naturalistic stimuli are processed and represented in the human brain remains a major challenge for psychology, neuroscience and computer vision. Current theoretical frameworks of object vision posit that effective recognition requires invariance -that mental representations remain stable despite changes in visual cues that can alter an object's appearance but not its identity 1-5 . Support for representational invariance has been derived from behavioral 6-8 and neuroimaging studies 3,9 (although with some limitations, for example, depending on the retinal location of the stimulus 10,11 and task demands 12 ). Two-dimensional (2-D) images of objects have been shown to elicit size-invariant responses during recognition 13 , priming 7,14 , perceptual learning 15 , and visual search 16 . Similarly, neuroimaging studies have shown that responses in shape-selective regions within ventral occipito-temporal (vOT) cortex remain relatively constant despite changes in image size 17,18 , as well as changes in position 19 , viewpoint 4,9,11,15 and depth cues 18,20 . Reliable adaptation in ventral object areas across changes in the size of object images is observed in children by 5 to 10 years of age 21 . FMRI responses in vOT have also been found to remain constant across changes in t...

show abstract

Section: Resultsmentioning

confidence: 99%

“…As in previous studies of image vision (e.g. [25][26][27][28]35 ), the 2-D images were scaled so that they had the same visual size.…”

Section: Resultsmentioning

confidence: 99%

Object responses are highly malleable, rather than invariant, with changes in object appearance

Holler

Fabbri

Snow

2020

Sci Rep

View full text Add to dashboard Cite

show abstract

“…What are the representational dimensions in other domains, such as words, faces, places, or actions? Finally, what makes those representations similar to those found in deep convolutional neural network models of vision 47 , semantic embeddings learned on word co-occurrence statistics in large text corpora 22,30,48 , or brain activity in humans [49][50][51][52][53] ? Addressing these questions will be important for a comprehensive understanding of mental representations of objects across people and different domains.…”

Section: Discussionmentioning

confidence: 98%

Revealing the multidimensional mental representations of natural objects underlying human similarity judgments

Hebart¹,

Zheng²,

Pereira³

et al. 2020

Preprint

View full text Add to dashboard Cite

Objects can be characterized according to a vast number of possible criteria (e.g. animacy, shape, color, function), but some dimensions are more useful than others for making sense of the objects around us. To identify these "core dimensions" of object representations, we developed a data-driven computational model of similarity judgments for real-world images of 1,854 objects. The model captured most explainable variance in similarity judgments and produced 49 highly reproducible and meaningful object dimensions that reflect various conceptual and perceptual properties of those objects. These dimensions predicted external categorization behavior and reflected typicality judgments of those categories. Further, humans can accurately rate objects along these dimensions, highlighting their interpretability and opening up a way to generate similarity estimates from object dimensions alone. Collectively, these results demonstrate that human similarity judgments can be captured by a fairly lowdimensional, interpretable embedding that generalizes to external behavior.

show abstract

“…Or do they partly reflect idiosyncratic preferences, that is an individual person's unique aesthetic preference for particular faces? To resolve this question, we performed an analysis where we modelled neural RDMs as a function of individual yes/no responses and attractiveness ratings, while controlling for the database ratings using partial correlation analysis [43][44][45] (Figure 2c). We found that individual attractiveness judgments still significantly predicted cortical representations, both when considering yes/no responses (from the 150-200ms time bin; peaking at 600-650ms, peak t [22]=3.87, p<0.001, pcorr=0.002) and when considering attractiveness ratings (from the 150-200ms time bin; peaking at 500-550ms, peak t [22]=3.90, p<0.001, pcorr=0.012).…”

Section: Early Representations Of Facial Attractiveness Reflect Indivmentioning

confidence: 99%

“…To establish correspondences between the neural RDM and a specific predictor RDM while controlling for other RDMs (see below), we used partial correlations [43][44][45]. All correlations were Fisher-transformed before entering them into statistical analyses.…”

Section: Tracking Neural Representations Of Facial Attractivenessmentioning

confidence: 99%

Tracking cortical representations of facial attractiveness using time-resolved representational similarity analysis

Kaiser

Nyga

2020

Preprint

View full text Add to dashboard Cite

When we see a face, we rapidly form an impression of its attractiveness. Here, we investigated how rapidly representations of facial attractiveness emerge in the human brain. In an EEG experiment, participants viewed 100 face photographs and rated them for their attractiveness. Using time-resolved representational similarity analysis on the EEG data, we reveal representations of facial attractiveness after 150-200ms of cortical processing. Interestingly, we show that these representations are related to individual participants' personal attractiveness judgments, suggesting that already early perceptual representations of facial attractiveness convey idiosyncratic attractiveness preferences. Further, we show that these early representations are genuinely related to attractiveness, as they are neither explained by other high-level face attributes, such as face sex or age, nor by features extracted by an artificial deep neural network model of face processing. Together, our results demonstrate early, individually specific, and genuine representations of facial attractiveness, which may underlie fast attractiveness judgments. IntroductionWhen we see a face, we almost immediately can tell whether we find it attractive or not [1]. Beyond such first impressions, facial attractiveness affects people's everyday lives in fundamental ways: for instance, an attractive face grants advantages in various aspects, such as increased success in dating [2,3], receiving help more often [4,5], and being more successful on the job market [6,7].Given these varied effects of facial attractiveness, a large body of research has focused on understanding the factors that make a face attractive. One line of research has tried to establish objective physical markers of facial attractiveness [8,9], revealing that faces are more attractive when they are more similar to the average [10,11], more symmetric [11][12][13], or have favourable sexual characteristics [14]. This research has led into advances in computer vision, providing algorithms that can predict how attractive humans will find a particular face [15,16]. However, there is considerable agreement that there is also a subjective component to facial attractiveness, with responses varying substantially between observers [17][18][19]. Indeed, our impression of facial attractiveness may depend on both: an objectively attractive physical composition of visual features and an idiosyncratic appreciation of these features.How does the brain achieve the transition from physical stimulus properties into an individual representation of facial attractiveness? To answer this question, previous studies have used event-related potentials (ERPs) obtained from EEG recordings to investigate when brain responses to more or less attractive faces differ.Although in many such studies facial attractiveness influences multiple ERP components, a key question is when facial attractiveness is first represented in EEG signals. The answers to this question are quite mixed. Some studies highlight relatively early ER...

show abstract

The spatiotemporal neural dynamics underlying perceived similarity for real-world objects

Cited by 66 publications

References 101 publications

Object responses are highly malleable, rather than invariant, with changes in object appearance

Object responses are highly malleable, rather than invariant, with changes in object appearance

Revealing the multidimensional mental representations of natural objects underlying human similarity judgments

Tracking cortical representations of facial attractiveness using time-resolved representational similarity analysis

Contact Info

Product

Resources

About