Classification and Geometry of General Perceptual Manifolds

Abstract

Perceptual manifolds arise when a neural population responds to an ensemble of sensory signals associated with different physical features (e.g., orientation, pose, scale, location, and intensity) of the same perceptual object. Object recognition and discrimination require classifying the manifolds in a manner that is insensitive to variability within a manifold. How neuronal systems give rise to invariant object classification and recognition is a fundamental problem in brain theory as well as in machine learning. Here, we study the ability of a readout network to classify objects from their perceptual manifold representations. We develop a statistical mechanical theory for the linear classification of manifolds with arbitrary geometry, revealing a remarkable relation to the mathematics of conic decomposition. We show how special anchor points on the manifolds can be used to define novel geometrical measures of radius and dimension, which can explain the classification capacity for manifolds of various geometries. The general theory is demonstrated on a number of representative manifolds, including l 2 ellipsoids prototypical of strictly convex manifolds, l 1 balls representing polytopes with finite samples, and ring manifolds exhibiting nonconvex continuous structures that arise from modulating a continuous degree of freedom. The effects of label sparsity on the classification capacity of general manifolds are elucidated, displaying a universal scaling relation between label sparsity and the manifold radius. Theoretical predictions are corroborated by numerical simulations using recently developed algorithms to compute maximum margin solutions for manifold dichotomies. Our theory and its extensions provide a powerful and rich framework for applying statistical mechanics of linear classification to data arising from perceptual neuronal responses as well as to artificial deep networks trained for object recognition tasks.

Separability and Geometry of Object Manifolds in Deep Neural Networks

“…Summary of manifold classification capacity Following the theory introduced in [16], manifolds are described by D + 1 coordinates, one of them defines the location of the manifold center and the others the axes of the manifold variability. The set of points that define the manifold within its subspace of variability is formally designated as S which can represent a collection of finite number of data points or a smooth manifold (e.g., sphere or a curve).…”

“…As separability of manifolds depends on numerous variables -size and shape of the manifolds, number of neurons, cardinality of the set of manifolds, specific target labels, among others -it has been difficult to elucidate which specific properties of the representation truly contribute to untangling. Here we apply the theory of linear separability of manifolds [16] to establish analytically that separability depends on three measurable quantities, manifold dimension and extent, and inter-manifold correlation.…”

“…We first illustrate the role of manifold geometry on classification capacity by considering several limiting extremes. The largest possible value of α c is 2 [17] [16] and is achieved for manifolds consisting of single points [18], i.e. fully invariant object representations.…”

“…Thus, the capacity of structured "point-cloud" manifolds consisting of M points per manifold obeys 2 M ≤ α c ≤ 2 (illustrated in figure 1a). Another illuminating limit is that of manifolds with infinite sizes, each spanning D randomly oriented axes of variation (illustrated in figure 1b), in which case α c = 1/(D + 1/2) [16].…”

“…Using statistical mechanical mean field techniques, we derive algorithms that allow measuring the capacity, R M and D M for manifolds given by either empirical data samples or from parametric generative models [16]. This mean field theory previously assumed that the position and orientation of different manifolds are uncorrelated.…”