Separability and Geometry of Object Manifolds in Deep Neural Networks

Abstract

Stimuli are represented in the brain by the collective population responses of sensory neurons, and an object presented under varying conditions gives rise to a collection of neural population responses called an "object manifold." Changes in the object representation along a hierarchical sensory system are associated with changes in the geometry of those manifolds, and recent theoretical progress connects this geometry with "classification capacity," a quantitative measure of the ability to support object classification. Deep neural networks trained on object classification tasks are a natural testbed for the applicability of this relation. We show how classification capacity improves along the hierarchies of deep neural networks with different architectures. We demonstrate that changes in the geometry of the associated object manifolds underlie this improved capacity, and shed light on the functional roles different levels in the hierarchy play to achieve it, through orchestrated reduction of manifolds' radius, dimensionality and inter-manifold correlations.

Transformation of population code from dLGN to V1 facilitates linear decoding

“…Overall, our results suggest that the dLGN-V1 transformation can make representations of simple visual features more linearly separable by reshaping correlated spiking variability, even without increasing their dimensionality. This finding complements recent work that has shown that changes in the geometry of visual representations of objects contribute to their increased separability in deep neural networks (Cohen et al, 2019;Recanatesi et al, 2019). However, an important caveat of our interpretation is that, because we have subsampled dLGN and V1 recordings the same population size, we may miss an increase in dimensionality that could occur due to the anatomical divergence.…”

Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future

“…High level concepts that are intertangled in the pixel or retinal representation get pulled apart to form easily separable clusters in later representations. This theory has been developed using biological data [85], however, recently, techniques for describing the geometry of these clusters (or manifolds) have been developed and used to understand the untangling process in deep neural networks [86,87]. This work highlights the relevant features of these manifolds for classification and how they change through learning and processing stages.…”

Strong and localized recurrence controls dimensionality of neural activity across brain areas

“…Several studies in deep and recurrent artificial neural networks have highlighted how dimensionality modulation (compression and expansion) in neural representations across network layers (6, 73) and stages of learning (7, 74, 75) have functional roles in information processing. We next compute dimensionality on a finer scale that for the regions studied above – here for areas that subdivide those regions – to test this idea in data from diverse neural circuits.…”

“…Such a trend is consistent with the hypothesis that the visual stream performs a stimulus-dependent dimensionality expansion, akin to the trend described in artificial neural networks and often explained in terms of feature expansion of the input, Figs. S11a to S11b (6,73,76). We note that (77) recently studied the related but distinct quantity of “object manifold dimensionality” computed across transformations of a visual object, in optical recordings from some of these same areas, and found distinct trends for that quantity that are also consistent with dimensionality playing a role in visual information processing.…”