How neurons encode natural stimuli is a fundamental question for sensory neuroscience. In the early visual system, standard encoding models assume that neurons linearly filter incoming stimuli through their receptive fields, but artificial stimuli, such as contrast-reversing gratings, often reveal nonlinear spatial processing. We investigated to what extent such nonlinear processing is relevant for the encoding of natural images in retinal ganglion cells in mice of either sex.

A central goal in sensory neuroscience is to understand the neuronal signal processing involved in the encoding of natural stimuli. A critical step towards this goal is the development of successful computational encoding models. For ganglion cells in the vertebrate retina, the development of satisfactory models for responses to natural visual scenes is an ongoing challenge. Standard models typically apply linear integration of visual stimuli over space, yet many ganglion cells are known to show nonlinear spatial integration, in particular when stimulated with contrast-reversing gratings. We here study the influence of spatial nonlinearities in the encoding of natural images by ganglion cells, using multielectrode-array recordings from isolated salamander and mouse retinas. We assess how responses to natural images depend on first- and second-order statistics of spatial patterns inside the receptive field. This leads us to a simple extension of current standard ganglion cell models. We show that taking not only the weighted average of light intensity inside the receptive field into account but also its variance over space can partly account for nonlinear integration and substantially improve response predictions of responses to novel images. For salamander ganglion cells, we find that response predictions for cell classes with large receptive fields profit most from including spatial contrast information. Finally, we demonstrate how this model framework can be used to assess the spatial scale of nonlinear integration. Our results underscore that nonlinear spatial stimulus integration translates to stimulation with natural images. Furthermore, the introduced model framework provides a simple, yet powerful extension of standard models and may serve as a benchmark for the development of more detailed models of the nonlinear structure of receptive fields.

The role of the vertebrate retina in early vision is generally described by the efficient coding theory, which predicts that the retina discards spatiotemporal correlations in natural scenes. It is unclear, however, whether the predicted decorrelation in the activity of ganglion cells, the retina's output neurons, holds under gaze shifts, which dominate the natural visual input. We here show that species-specific gaze patterns in natural stimuli can drive strong and correlated spiking responses both within and across distinct types of ganglion cells in marmoset as well as mouse retina. These concerted responses violate efficient coding and signal fixation periods with locally high spatial contrast. Finally, novel model-based analyses of ganglion cell responses to natural stimuli reveal that the observed response correlations follow from nonlinear pooling of ganglion cell inputs. Our results reveal how concerted population activity can surpass efficient coding to detect gaze-related stimulus features.