Comparison of Object Recognition Behavior in Human and Monkey

Rajalingham, Rishi; Schmidt, Kailyn; DiCarlo, James J.

doi:10.1167/14.10.191

Cited by 21 publications

(41 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The behavioral data used in the current round of benchmarks was obtained by Rajalingham et al (2015) and . Here we focus on only the human behavioral data, but the human and nonhuman primate behavioral patterns are very similar to each other (Rajalingham et al, 2015. The image set used in this data collection was generated in a similar way as the images for V4 and IT using 24 object categories.…”

Section: Primate Behavioral Datamentioning

confidence: 99%

“…Behavioral ceilings, on the other hand, might not be prone to such ceiling effects as they are already estimated using multiple humans responses (i.e. the "pooled" human data, see Rajalingham et al (2015). However, reaching consistency with the pooled human behavioral may not be the only way that one might want to use ANN models to inform brain science, as the across-subject variation is also an important aspect of the data that models should aim to inform on.…”

Section: By Acquiring the Same Types Of Data Using New Imagesmentioning

confidence: 99%

See 1 more Smart Citation

Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?

Schrimpf¹,

Kubilius

Hong

et al. 2018

Preprint

377

578

View full text Add to dashboard Cite

The internal representations of early deep artificial neural networks (ANNs) were found to be remarkably similar to the internal neural representations measured experimentally in the primate brain. Here we ask, as deep ANNs have continued to evolve, are they becoming more or less brain-like? ANNs that are most functionally similar to the brain will contain mechanisms that are most like those used by the brain. We therefore developed Brain-Score -a composite of multiple neural and behavioral benchmarks that score any ANN on how similar it is to the brain's mechanisms for core object recognition -and we deployed it to evaluate a wide range of state-of-the-art deep ANNs. Using this scoring system, we here report that: (1) DenseNet-169, CORnet-S and ResNet-101 are the most brain-like ANNs.(2) There remains considerable variability in neural and behavioral responses that is not predicted by any ANN, suggesting that no ANN model has yet captured all the relevant mechanisms.(3) Extending prior work, we found that gains in ANN ImageNet performance led to gains on Brain-Score. However, correlation weakened at ≥ 70% top-1 ImageNet performance, suggesting that additional guidance from neuroscience is needed to make further advances in capturing brain mechanisms. (4) We uncovered smaller (i.e. less complex) ANNs that are more brain-like than many of the best-performing ImageNet models, which suggests the opportunity to simplify ANNs to better understand the ventral stream. The scoring system used here is far from complete. However, we propose that evaluating and tracking model-benchmark correspondences through a Brain-Score that is regularly updated with new brain data is an exciting opportunity: experimental benchmarks can be used to guide machine network evolution, and machine networks are mechanistic hypotheses of the brain's network and thus drive next experiments. To facilitate both of these, we release Brain-Score.org: a platform that hosts the neural and behavioral benchmarks, where ANNs for visual processing can be submitted to receive a Brain-Score and their rank relative to other models, and where new experimental data can be naturally incorporated. computational neuroscience | object recognition | deep neural networks

show abstract

Section: Primate Behavioral Datamentioning

confidence: 99%

Section: By Acquiring the Same Types Of Data Using New Imagesmentioning

confidence: 99%

Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?

Schrimpf¹,

Kubilius

Hong

et al. 2018

Preprint

377

578

View full text Add to dashboard Cite

show abstract

“…22 Interestingly, further developments of these early computational models have led to modern deep convolutional neural networks (DCNNs), which have powered recent breakthroughs in computer vision 23 as well as many other domains. Although these network models are not constrained by experimental data, they have nonetheless been shown to provide an even better fit than earlier models to both behavioral 18,24,25 and electrophysiological 26,27 data (but see Ref. 28).…”

Section: Introductionmentioning

confidence: 96%

“…7,16 This pattern of correct and incorrect answers suggests an underlying visual strategy implemented in the bottom-up phase, which appears to be largely shared between human and nonhuman primates. 14,17,18 Starting with Fukushima's neocognitron, 19 computational models constrained by the anatomy and physiology of the visual cortex (VC) (see Refs. 20-22 for reviews) account relatively well for this pattern of behavioral responses.…”

Section: Introductionmentioning

confidence: 99%

Beyond the feedforward sweep: feedback computations in the visual cortex

Kreiman

Serre

2020

Annals of the New York Academy of Sciences

View full text Add to dashboard Cite

Visual perception involves the rapid formation of a coarse image representation at the onset of visual processing, which is iteratively refined by late computational processes. These early versus late time windows approximately map onto feedforward and feedback processes, respectively. State‐of‐the‐art convolutional neural networks, the main engine behind recent machine vision successes, are feedforward architectures. Their successes and limitations provide critical information regarding which visual tasks can be solved by purely feedforward processes and which require feedback mechanisms. We provide an overview of recent work in cognitive neuroscience and machine vision that highlights the possible role of feedback processes for both visual recognition and beyond. We conclude by discussing important open questions for future research.

show abstract

“…Differences between human and macaque visual system The monkey visual system as model for the human visual system has been validated under several different aspects [37]. While the human visual system is from an anatomical and physiological perspective extremely similar to the macaque visual system, it has a slightly higher retinal magnification factor (about 0.291/0.223), which hints to a higher angular resolution [28].…”

Section: The Spatial Frequency ωmentioning

confidence: 99%

Linear Subspaces of the Appearance Space

Mylo

Klein

2018

JWSCG

View full text Add to dashboard Cite

Manipulating digital optical material representations is still a difficult problem because arbitrary manipulations lead almost certainly to an unrealistic impression of the material. In this paper we present an approach to material editing based on a digital model of the V1-area of the visual cortex. The V1-model is used to define the appearance space as the space of weighted sums of the cortical-model filter responses. We will show that it is possible to transform several optical material manipulation schemes into our editing scheme. As those optical material manipulation schemes may also be physical phenomena, we may introduce a new material edit. Our argumentation will be supported by comparing editing-examples.

show abstract

Comparison of Object Recognition Behavior in Human and Monkey

Cited by 21 publications

References 0 publications

Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?

Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?

Beyond the feedforward sweep: feedback computations in the visual cortex

Linear Subspaces of the Appearance Space

Contact Info

Product

Resources

About