Population response magnitude variation in inferotemporal cortex predicts image memorability

Jaegle, Andrew; Mehrpour, Vahid; Mohsenzadeh, Yalda; Meyer, Travis; Oliva, Aude; Rust, Nicole C.

doi:10.7554/elife.47596

Cited by 61 publications

(61 citation statements)

References 23 publications

(50 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Right, Behavioral responses triggered by changes in optogenetic input reflect the integrated spike counts, as behavioral responses were always suppressed by reductions in spiking regardless of whether reductions were followed by rapid release from inhibition. (Jaegle et al, 2019). These results strongly support the idea that decoding by central visual structures depends on increases in spiking.…”

Section: Discussionsupporting

confidence: 79%

Mice Preferentially Use Increases in Cerebral Cortex Spiking to Detect Changes in Visual Stimuli

et al. 2020

View full text Add to dashboard Cite

Whenever the retinal image changes, some neurons in visual cortex increase their rate of firing whereas others decrease their rate of firing. Linking specific sets of neuronal responses with perception and behavior is essential for understanding mechanisms of neural circuit computation. We trained mice of both sexes to perform visual detection tasks and used optogenetic perturbations to increase or decrease neuronal spiking primary visual cortex (V1). Perceptual reports were always enhanced by increments in V1 spike counts and impaired by decrements, even when increments and decrements in spiking were generated in the same neuronal populations. Moreover, detecting changes in cortical activity depended on spike count integration rather than instantaneous changes in spiking. Recurrent neural networks trained in the task similarly relied on increments in neuronal activity when activity has costs. This work clarifies neuronal decoding strategies used by cerebral cortex to translate cortical spiking into percepts that can be used to guide behavior.

show abstract

Section: Discussionsupporting

confidence: 79%

Mice Preferentially Use Increases in Cerebral Cortex Spiking to Detect Changes in Visual Stimuli

et al. 2020

View full text Add to dashboard Cite

show abstract

“…However, the question of which stimulus properties are easier to remember has been rapidly growing in popularity in recent years. Studies that focused on the concept of intrinsic memorability have typically used a combination of visual factors to predict subsequent memory of scenes (Bainbridge et al, 2017;Isola et al, 2014), objects (Jaegle et al, 2019), and unfamiliar faces (Bainbridge et al, 2013). Very few such memorability studies have examined verbal stimuli, and the semantic factors examined in such studies are usually limited to category membership (Bainbridge & Rissman, 2018) or automatic labels generated by automated computer vision algorithms (Borkin et al, 2016;Isola et al, 2014).…”

Section: Introductionmentioning

confidence: 99%

The visual and semantic features that predict object memory: Concept property norms for 1,000 object images

et al. 2021

View full text Add to dashboard Cite

Humans have a remarkable fidelity for visual long-term memory, and yet the composition of these memories is a longstanding debate in cognitive psychology. While much of the work on long-term memory has focused on processes associated with successful encoding and retrieval, more recent work on visual object recognition has developed a focus on the memorability of specific visual stimuli. Such work is engendering a view of object representation as a hierarchical movement from low-level visual representations to higher level categorical organization of conceptual representations. However, studies on object recognition often fail to account for how these high- and low-level features interact to promote distinct forms of memory. Here, we use both visual and semantic factors to investigate their relative contributions to two different forms of memory of everyday objects. We first collected normative visual and semantic feature information on 1,000 object images. We then conducted a memory study where we presented these same images during encoding (picture target) on Day 1, and then either a Lexical (lexical cue) or Visual (picture cue) memory test on Day 2. Our findings indicate that: (1) higher level visual factors (via DNNs) and semantic factors (via feature-based statistics) make independent contributions to object memory, (2) semantic information contributes to both true and false memory performance, and (3) factors that predict object memory depend on the type of memory being tested. These findings help to provide a more complete picture of what factors influence object memorability. These data are available online upon publication as a public resource.

show abstract

“…In addition to insights into the nature of these differences, this is of importance to the neuroscience community as different distance measures emphasize different aspects of the data that could be of specific interest in a given experimental setting. For example, the vector angle of neural population responses in IT was recently shown to encode different cognitive aspects than their magnitude 24 . This requires computational models to be probed in the same manner.…”

Section: Discussionmentioning

confidence: 99%

Individual differences among deep neural network models

et al. 2020

View full text Add to dashboard Cite

Deep neural networks (DNNs) excel at visual recognition tasks and are increasingly used as a modeling framework for neural computations in the primate brain. Just like individual brains, each DNN has a unique connectivity and representational profile. Here, we investigate individual differences among DNN instances that arise from varying only the random initialization of the network weights. Using tools typically employed in systems neuroscience, we show that this minimal change in initial conditions prior to training leads to substantial differences in intermediate and higher-level network representations despite similar network-level classification performance. We locate the origins of the effects in an under-constrained alignment of category exemplars, rather than misaligned category centroids. These results call into question the common practice of using single networks to derive insights into neural information processing and rather suggest that computational neuroscientists working with DNNs may need to base their inferences on groups of multiple network instances.

show abstract

Population response magnitude variation in inferotemporal cortex predicts image memorability

Cited by 61 publications

References 23 publications

Mice Preferentially Use Increases in Cerebral Cortex Spiking to Detect Changes in Visual Stimuli

Mice Preferentially Use Increases in Cerebral Cortex Spiking to Detect Changes in Visual Stimuli

The visual and semantic features that predict object memory: Concept property norms for 1,000 object images

Individual differences among deep neural network models

Contact Info

Product

Resources

About