Normative theory of visual receptive fields

Abstract

This article gives an overview of a normative theory of visual receptive fields. We describe how idealized functional models of early spatial, spatio-chromatic and spatio-temporal receptive fields can be derived in a principled way, based on a set of axioms that reflect structural properties of the environment in combination with assumptions about the internal structure of a vision system to guarantee consistent handling of image representations over multiple spatial and temporal scales. Interestingly, this theory leads to predictions about visual receptive field shapes with qualitatively very good similarities to biological receptive fields measured in the retina, the LGN and the primary visual cortex (V1) of mammals.

A time-causal and time-recursive scale-covariant scale-space representation of temporal signals and past time

“…Temporal smoothing kernels of this form, combined with temporal differentiation for different orders of differentiation, to obtain ripples of opposite contrast in the resulting temporal receptive fields, have been used for modelling the temporal part of the processing in models for spatio-temporal receptive fields (Lindeberg and Fagerström 1996;Lindeberg 2015Lindeberg , 2016Lindeberg , 2021b and spatio-temporal receptive fields (Lindeberg and Friberg 2015b, 2015a).…”

“…These kernels have been used as temporal basis functions for spatio-temporal receptive fields (Lindeberg 2016(Lindeberg , 2021bJansson and Lindeberg 2018) and for expressing methods for temporal scale selection (Lindeberg 2017(Lindeberg , 2018b and spatio-temporal scale selection (Lindeberg 2018a(Lindeberg , 2018b that detect and compare temporal structures at different temporal scales in a completely scale-invariant manner.…”

“…The subject of this article is to describe a theoretical framework for representing temporal signals at multiple temporal scales, intended for a more general audience without background in these areas and with the focus on the tem-poral domain only, thus without the complementary spatial and spectral domains that this theory has previously been combined with for expressing spatio-temporal and spectrotemporal receptive fields (Lindeberg and Fagerström 1996;Lindeberg 1997Lindeberg , 2016Lindeberg , 2017Lindeberg , 2018aLindeberg , 2018bLindeberg , 2021bFriberg 2015b, 2015a). This theoretical framework, referred to as temporal scale-space theory, guarantees noncreation of the temporal structures with increasing temporal scales, in the sense that it ensures that a temporal representation at any coarser temporal scale constitutes a simplification of an temporal representation at any finer temporal scale, in the respect that the number of local temporal extrema, alternatively the number of temporal zero-crossings, is guaranteed to not increase from finer to coarser temporal scales.…”

Scale-Covariant and Scale-Invariant Gaussian Derivative Networks

“…Approaches to handling image transformations in deep networks have also been developed based on formalism from group theory [33,[75][76][77]. A general framework for handling basic types of natural image transformations in terms of spatial scaling transformations, spatial affine transformations, Galilean transformations and temporal scaling transformations in the first layer of visual processing based on linear receptive fields and with relations to biological vision has been presented in [78], based on generalized axiomatic scalespace theory [12].…”

“…The corresponding affine extension of such receptive fields, by replacing the rotationally symmetric Gaussian kernel (5) for scale-space smoothing by a corresponding affine Gaussian kernel, does also constitute a good idealized model for the receptive fields of simple cells in the primary visual cortex [66,78]. In this treatment, we will, however, for simplicity restrict ourselves to regular Gaussian derivatives, based on partial derivatives and directional derivatives of rotationally symmetric Gaussian kernels.…”