Can Deep Neural Networks Rival Human Ability to Generalize in Core Object Recognition?

Kubilius, Jonas; Kar, Kohitij; Schmidt, Kailyn; DiCarlo, James J.

doi:10.32470/ccn.2018.1234-0

Cited by 6 publications

(11 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is also the case with CNNs widely used in computer science that are often described as the best current theory of object recognition in humans (e.g. Kubilius, Kar, Schmidt, & DiCarlo, 2018 ). Although the design of CNNs (both the convolution and the pooling layers) are claimed to support translation invariance (for introduction see O'Shea & Nash, 2015 ), these models are generally trained to categorize images by training multiple exemplars of each image category at multiple spatial locations.…”

Section: Introductionmentioning

confidence: 93%

The human visual system and CNNs can both support robust online translation tolerance following extreme displacements

et al. 2021

View full text Add to dashboard Cite

Visual translation tolerance refers to our capacity to recognize objects over a wide range of different retinal locations. Although translation is perhaps the simplest spatial transform that the visual system needs to cope with, the extent to which the human visual system can identify objects at previously unseen locations is unclear, with some studies reporting near complete invariance over 10 degrees and other reporting zero invariance at 4 degrees of visual angle. Similarly, there is confusion regarding the extent of translation tolerance in computational models of vision, as well as the degree of match between human and model performance. Here, we report a series of eye-tracking studies (total N = 70) demonstrating that novel objects trained at one retinal location can be recognized at high accuracy rates following translations up to 18 degrees. We also show that standard deep convolutional neural networks (DCNNs) support our findings when pretrained to classify another set of stimuli across a range of locations, or when a global average pooling (GAP) layer is added to produce larger receptive fields. Our findings provide a strong constraint for theories of human vision and help explain inconsistent findings previously reported with convolutional neural networks (CNNs).

show abstract

Section: Introductionmentioning

confidence: 93%

The human visual system and CNNs can both support robust online translation tolerance following extreme displacements

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Presently, our datasets use naturalistic images, generated by pasting objects on a random backgrounds. While these datasets are already extremely challenging, we will more stringently be able to test model ability to generalize beyond its training set by expanding our datasets to more classes of images (e.g., photographs, distorted images (Geirhos et al, 2018), artistic renderings (Kubilius et al, 2018a), images optimized for neural responses ).…”

Section: By Acquiring the Same Types Of Data Using New Imagesmentioning

confidence: 99%

Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?

Schrimpf¹,

Kubilius

Hong

et al. 2018

Preprint

Self Cite

410

578

View full text Add to dashboard Cite

The internal representations of early deep artificial neural networks (ANNs) were found to be remarkably similar to the internal neural representations measured experimentally in the primate brain. Here we ask, as deep ANNs have continued to evolve, are they becoming more or less brain-like? ANNs that are most functionally similar to the brain will contain mechanisms that are most like those used by the brain. We therefore developed Brain-Score -a composite of multiple neural and behavioral benchmarks that score any ANN on how similar it is to the brain's mechanisms for core object recognition -and we deployed it to evaluate a wide range of state-of-the-art deep ANNs. Using this scoring system, we here report that: (1) DenseNet-169, CORnet-S and ResNet-101 are the most brain-like ANNs.(2) There remains considerable variability in neural and behavioral responses that is not predicted by any ANN, suggesting that no ANN model has yet captured all the relevant mechanisms.(3) Extending prior work, we found that gains in ANN ImageNet performance led to gains on Brain-Score. However, correlation weakened at ≥ 70% top-1 ImageNet performance, suggesting that additional guidance from neuroscience is needed to make further advances in capturing brain mechanisms. (4) We uncovered smaller (i.e. less complex) ANNs that are more brain-like than many of the best-performing ImageNet models, which suggests the opportunity to simplify ANNs to better understand the ventral stream. The scoring system used here is far from complete. However, we propose that evaluating and tracking model-benchmark correspondences through a Brain-Score that is regularly updated with new brain data is an exciting opportunity: experimental benchmarks can be used to guide machine network evolution, and machine networks are mechanistic hypotheses of the brain's network and thus drive next experiments. To facilitate both of these, we release Brain-Score.org: a platform that hosts the neural and behavioral benchmarks, where ANNs for visual processing can be submitted to receive a Brain-Score and their rank relative to other models, and where new experimental data can be naturally incorporated. computational neuroscience | object recognition | deep neural networks

show abstract

“…The quantitative relationship of this axis to the real-world size of objects was evaluated with a series of functions in different scales, 𝐿𝑖𝑛𝑒𝑎𝑟: 𝑦 = 𝑥 (6) 𝐸𝑥𝑝𝑜𝑛𝑒𝑛𝑡𝑖𝑎𝑙: 𝑦 = 𝑥 $. ** (7) 𝐸𝑥𝑝𝑜𝑛𝑒𝑛𝑡𝑖𝑎𝑙: 𝑦 = 𝑥 $.+ (8) 𝐸𝑥𝑝𝑜𝑛𝑒𝑛𝑡𝑖𝑎𝑙: 𝑦 = 𝑥 % (9) 𝐸𝑥𝑝𝑜𝑛𝑒𝑛𝑡𝑖𝑎𝑙: 𝑦 = 𝑥 * (10) 𝐿𝑜𝑔𝑎𝑟𝑖𝑡ℎ𝑚𝑖𝑐: 𝑦 = ln 𝑥 (11) where x is the value of the real-world size PC, and y is the measured real-world size of objects. The scale with the best fit was taken as the relationship that best describes the data.…”

Section: Evaluate the Role Of Real-world Size In Object Spacementioning

confidence: 99%

“…Recently, deep convolutional neural networks (DCNNs) show great potentials of simulating primates' ventral visual pathway on object recognition [9][10][11][12][13] , with similar representations between DCNNs and primates been revealed (e.g., retinotopy 14 , semantic structure 15 , coding scheme 16 , and face representation 17 ).…”

Section: Introductionmentioning

confidence: 99%

Real-world size of objects serves as an axis of object space

Huang

Song

Liu

2021

Preprint

View full text Add to dashboard Cite

Our mind can represent various objects from the physical world metaphorically into an abstract and complex high-dimensional object space, with a finite number of orthogonal axes encoding critical object features. However, little is known about what features serve as axes of the object space to critically affect object recognition. Here we asked whether the feature of objects’ real-world size constructed an axis of object space with deep convolutional neural networks (DCNNs) based on three criteria of sensitivity, independence and necessity that are impractical to be examined altogether with traditional approaches. A principal component analysis on features extracted by the DCNNs showed that objects’ real-world size was encoded by an independent axis, and the removal of this axis significantly impaired DCNN’s performance in recognizing objects. With a mutually-inspired paradigm of computational modeling and biological observation, we found that the shape of objects, rather than retinal size, co-occurrence, task demands and texture features, was necessary to represent the real-world size of objects for DCNNs and humans. In short, our study provided the first evidence supporting the feature of objects’ real-world size as an axis of object space, and devised a novel paradigm for future exploring the structure of object space.

show abstract

Can Deep Neural Networks Rival Human Ability to Generalize in Core Object Recognition?

Cited by 6 publications

References 10 publications

The human visual system and CNNs can both support robust online translation tolerance following extreme displacements

The human visual system and CNNs can both support robust online translation tolerance following extreme displacements

Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?

Real-world size of objects serves as an axis of object space

Contact Info

Product

Resources

About