Creatures great and small: Real-world size of animals predicts visual cortex representations beyond taxonomic category

Coutanche, Marc N.; Koch, Griffin E.

doi:10.1016/j.neuroimage.2018.08.066

Cited by 13 publications

(12 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…VT is implicated in the representation of higher-level visual and conceptual features within multi-voxel patterns (Coutanche, Solomon, & Thompson-Schill, 2016;Haxby et al, 2001), including taxonomy-and species-relevant information for animal stimuli (Connoly et al, 2012;Coutanche and Koch, 2018). We defined VT within individuals based on a procedure outlined in Coutanche and Koch (2018) in which the region is defined as extending from 20 to 70 mm posterior to the anterior commissure, incorporating the lingual, fusiform, parahippocampal and inferior temporal gyri. We defined white matter by extracting the left/right white matter regions from the Harvard-Oxford subcortical structural atlases (Frazier et al, 2005).…”

Section: Imaging Acquisition and Preprocessingmentioning

confidence: 99%

Representational Connectivity Analysis: Identifying Networks of Shared Changes in Representational Strength through Jackknife Resampling

Coutanche

Buckser

Akpan

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Acknowledgments:We thank Mac Shine for invaluable discussions during the conception of this project, and thank Heather Bruett, Griffin Koch, John Paulus and Xueying Ren for conversations related to the work. We are also grateful to Samuel Nastase and co-authors for making their dataset available. AbstractThe structure of information in the brain is crucial to cognitive function. The representational space of a brain region can be identified through Representational Similarity Analysis (RSA) applied to functional magnetic resonance imaging (fMRI) data. In its classic form, RSA collapses the time-series of each condition, eliminating fluctuations in similarity over time. We propose a method for identifying representational connectivity (RC) networks, which share fluctuations in representational strength, in an analogous manner to functional connectivity (FC), which tracks fluctuations in BOLD signal, and informational connectivity, which tracks fluctuations in pattern discriminability. We utilize jackknife resampling, a statistical technique in which observations are removed in turn to determine their influence. We applied the jackknife technique to an existing fMRI dataset collected as participants viewed videos of animals (Nastase et al., 2017). We used ventral temporal cortex (VT) as a seed region, and compared the resulting network to a second-order RSA, in which brain regions' representational spaces are compared, and to the network identified through FC. The novel representational connectivity analysis identified a network comprising regions associated with lower-level visual processing, spatial cognition, perceptual-motor integration, and visual attention, indicating that these regions shared fluctuations in representational similarity strength with VT. RC, second-order RSA and FC identified areas unique to each method, indicating that analyzing shared fluctuations in the strength of representational similarity reveals previously undetectable networks of regions. The RC analysis thus offers a new way to understand representational similarity at the network level.

show abstract

Section: Imaging Acquisition and Preprocessingmentioning

confidence: 99%

Representational Connectivity Analysis: Identifying Networks of Shared Changes in Representational Strength through Jackknife Resampling

Coutanche

Buckser

Akpan

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, these evaluations are still focused on creating variability within a dataset, offering no guarantee that the network is not simply overfit to it. Moving beyond datasets, our proposed evaluation metric HMS quantifies consistency in a network's internal behavior by directly comparing against the internal behavior of one of the most generalizable vision systems in the world: the biological brain [7,14,35].…”

Section: Related Workmentioning

confidence: 99%

“…However, the ability of biological beings to generalize and adapt extends from both structure and internal behavior. Internally, the visual system processes similar objects with similar patterns of cell activations [7,14,35]. This activation behavior is an observable manifestation of the brain's ability to generalize beyond its experience, such as automatically allowing the classification of unseen instances of object classes (e.g., correctly identifying a car despite never having seen this specific car before).…”

Section: Introductionmentioning

confidence: 99%

A Neurobiological Evaluation Metric for Neural Network Model Search

Blanchard

Kinnison

RichardWebster

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Neuroscience theory posits that the brain's visual system coarsely identifies broad object categories via neural activation patterns, with similar objects producing similar neural responses. Artificial neural networks also have internal activation behavior in response to stimuli. We hypothesize that networks exhibiting brain-like activation behavior will demonstrate brain-like characteristics, e.g., stronger

show abstract

“…In another relevant study, Gabay and colleagues trained participants through extensive exposure to geometric shapes of different sizes, finding that early visual cortex activation was stronger for shapes that had previously been associated with larger sizes (Gabay et al, 2016). Finally, in a recent study of how real-world size is processed in visual cortex, Coutanche and Koch (2018) found that size was represented in early visual cortex beyond taxonomic category. By using animal species that break the typical correlation between real-world size and taxonomic category (e.g., insects that are bigger than birds, and birds that are bigger than mammals), the authors found pattern similarity based on real-world size after accounting for taxonomic and visual differences.…”

Section: Introductionmentioning

confidence: 99%

“…By using animal species that break the typical correlation between real-world size and taxonomic category (e.g., insects that are bigger than birds, and birds that are bigger than mammals), the authors found pattern similarity based on real-world size after accounting for taxonomic and visual differences. Thus, when examining how the brain responds to visually presented items that are neither manipulable nor potential landmarks, real-world size information has been found in early visual cortex (Borghesani et al, 2016;Coutanche & Koch, 2018;Gabay et al, 2016). In some cases, this is accompanied by an absence of size information in VT cortex for these same items (Borghesani et al, 2016;Coutanche & Koch, 2018).…”

Section: Introductionmentioning

confidence: 99%

Neural activity in human visual cortex is transformed by learning real world size

Coutanche

Thompson-Schill

2019

NeuroImage

Self Cite

View full text Add to dashboard Cite

The way that our brain processes visual information is directly affected by our experience. Repeated exposure to a visual stimulus triggers experience-dependent plasticity in the visual cortex of many species. Humans also have the unique ability to acquire visual knowledge through instruction. We introduced human participants to the real-world size of previously unfamiliar species, and to the functional motion of novel tools, during a functional magnetic resonance imaging scan. Using machine learning, we compared activity patterns evoked by images of the new items, before and after participants learned the animals' real-world size or tools' motion. We found that, after acquiring size information, participants' visual activity patterns became more confusable between novel and same-sized animals in early visual cortex, but not in ventral temporal cortex, reflecting an influence of new size knowledge on posterior, but not anterior, components of the ventral stream. In contrast, learning the functional motion of new tools did not lead to an equivalent change in recorded activity. Finally, the time-points marked by evidence of new size information in early visual cortex were more likely to show size information and greater activation in the right angular gyrus, a key hub of semantic knowledge and spatial cognition. Overall, these findings suggest that learning an item's real-world size by instruction influences subsequent activity in visual cortex and in a region that is central to semantic and spatial brain systems.

show abstract

Creatures great and small: Real-world size of animals predicts visual cortex representations beyond taxonomic category

Cited by 13 publications

References 22 publications

Representational Connectivity Analysis: Identifying Networks of Shared Changes in Representational Strength through Jackknife Resampling

Representational Connectivity Analysis: Identifying Networks of Shared Changes in Representational Strength through Jackknife Resampling

A Neurobiological Evaluation Metric for Neural Network Model Search

Neural activity in human visual cortex is transformed by learning real world size

Contact Info

Product

Resources

About