Unsupervised Neural Network Models of the Ventral Visual Stream

Abstract

Deep neural networks currently provide the best quantitative models of the response patterns of neurons throughout the primate ventral visual stream. However, such networks have remained implausible as a model of the development of the ventral stream, in part because they are trained with supervised methods requiring many more labels than are accessible to infants during development. Here, we report that recent rapid progress in unsupervised learning has largely closed this gap. We find that neural network models learned with deep unsupervised contrastive embedding methods achieve neural prediction accuracy in multiple ventral visual cortical areas that equals or exceeds that of models derived using today’s best supervised methods, and that the mapping of these neural network models’ hidden layers is neuroanatomically consistent across the ventral stream. Moreover, we find that these methods produce brain-like representations even when trained on noisy and limited data measured from real children’s developmental experience. We also find that semi-supervised deep contrastive embeddings can leverage small numbers of labelled examples to produce representations with substantially improved error-pattern consistency to human behavior. Taken together, these results suggest that deep contrastive embedding objectives may be a biologically-plausible computational theory of primate visual development.

An ecologically motivated image dataset for deep learning yields better models of human vision

“…We here trained all DNNs to optimize for categorization performance. While this task is undoubtedly of ecological relevance, the explanatory power of unsupervised objectives ( 35 – 37 ), semantically better-informed training targets, and their interplay with ecoset will be worth considering going forward.…”

Learning from unexpected events in the neocortical microcircuit

“…Theoretical support for this hypothesis comes from computational studies showing that hierarchical models that learn by comparing top-down signals to bottom-up signals enable artificial neural networks (ANNs) to learn useful representations that capture the statistical structure of the data on which they are trained [Lotter et al, 2016; Devlin et al, 2018; van den Oord et al, 2018; Grill et al, 2020; Chen et al, 2020; Wayne et al, 2018]. Moreover, ANNs trained in this manner reproduce the representations observed in the neocortex better than ANNs trained purely by supervised learning based on categorical labels [Konkle and Alvarez, 2020; Zhuang et al, 2020; Higgins et al, 2017; Christensen and Zylberberg, 2020].…”

SAYCam: A Large, Longitudinal Audiovisual Dataset Recorded From the Infant’s Perspective

“…Finally, the density of our dataset is extremely important for the use of new neural network algorithms, which tend to be "data hungry." Already, researchers have explored what learning progress is possible when new unsupervised visual learning algorithms are applied to our dataset (Orhan et al, 2020;Zhuang et al, 2020). In fact, as Orhan et al (2020) note, even though our dataset is quite large by conventional standards, a child's visual experience from birth to age two and a half would in fact be two orders of magnitude larger still, and the machine learning literature suggests that such an increase in data would be likely to lead to higher performance and new innovations.…”