Neural Encoding of Scene Statistics for Surface and Object Inference

Abstract

Abstract:Features associated with an object or its surfaces in natural scenes tend to vary coherently in space and time. In psychological literature, these coherent covariations have been considered to be important for neural systems to acquire models of objects and object categories. From a statistical inference perspective, such coherent covariation can provide a mechanism to learn the statistical priors in natural scenes that are useful for probabilistic inference. In this article, we present some neurophysiological experimental observations in the early visual cortex that provide insights on how correlation structures in visual scenes are being encoded by neuronal tuning and connections among neurons. The key insight is that correlated structures in visual scenes result in correlated neuronal activities, which shapes the tuning properties of individual neurons and the connections between them, embedding Gestalt-related computational constraints or priors for surface inference. Extending these concepts to the inferotemporal cortex suggests a representational framework that is distinct from traditional feed-forward hierarchy of invariant object representation and recognition. In this framework, lateral connections among view-based neurons, learned from the temporal association of the object views observed over time, can form a linked graph structure with local dependency, akin to a dense aspect graph in computer vision. This web-like graph allows view-invariant object representation to be created using sparse feed-forward connections, while maintaining the explicit representation of the different views. Thus, it can serve as an effective prior model for generating predictions of future incoming views to facilitate object inference.

Saccades during Object Viewing Modulate Oscillatory Phase in the Superior Temporal Sulcus

“…On the contrary, a hysteresis, or neural "memory" between fixations may be adaptive for transsaccadic integration (Melcher, 2005) and context-dependent processing (Buonomano and Maass, 2009). Given natural image statistics, saccade planning could also involve predictions about the chromatic and high spatial frequency content of a peripheral object fragment, given the current foveal image and the peripheral view of the impending saccade target (Rao and Ballard, 1999;Lee and Mumford, 2003;Lee et al, 2009). Indeed, distinct IT neuronal responses for two objects converge following pairing of the peripheral view of one object with the foveal view of the other, "fooling" the visual system by swapping object identity during saccades (Li and DiCarlo, 2008).…”