An invariance principle for maintaining the operating point of a neuron

Abstract

Sensory neurons adapt to changes in the natural statistics of their environments through processes such as gain control and firing threshold adjustment. It has been argued that neurons early in sensory pathways adapt according to information-theoretic criteria, perhaps maximising their coding efficiency or information rate. Here, we draw a distinction between how a neuron's preferred operating point is determined and how its preferred operating point is maintained through adaptation. We propose that a neuron's preferred operating point can be characterised by the probability density function (PDF) of its output spike rate, and that adaptation maintains an invariant output PDF, regardless of how this output PDF is initially set. Considering a sigmoidal transfer function for simplicity, we derive simple adaptation rules for a neuron with one sensory input that permit adaptation to the lower-order statistics of the input, independent of how the preferred operating point of the neuron is set. Thus, if the preferred operating point is, in fact, set according to information-theoretic criteria, then these rules nonetheless maintain a neuron at that point. Our approach generalises from the unimodal case to the multimodal case, for a neuron with inputs from distinct sensory channels, and we briefly consider this case too.

Sparseness, Antisparseness and Anything in Between: The Operating Point of a Neuron Determines Its Computational Repertoire

“…but we prefer the equivalent although somewhat more intuitive parametrisation (Elliott et al, 2008) r…”

“…The parameters Γ and Θ are constants intrinsic to the neuron that determine the neuron's preferred operating point. For more general input statistics, these rules will result in only approximate invariance, but we suggested that real neurons, in the face of intrinsic noise and stochasticity, may not need to maintain precise invariance and that these rules may be good enough for most practical purposes (Elliott et al, 2008).…”

“…In previous work, we proposed a phenomenological adaptation principle that allows a neuron to maintain an (approximately) invariant output firing rate PDF in the face of changing input statistics (Elliott et al, 2008). We called this invariant output firing rate PDF the neuron's "operating point".…”

“…Triesch's mechanism pushes a neuron into a large θ regime in order to achieve sparseness, but these dynamics are largely insensitive to the precise details of the input statistics, at least for centred, whitened, ICA-like inputs. While our own mechanism is constructed specifically to permit adaptation to changing input statistics (Elliott et al, 2008), including the generalisation proposed here of adapting to changes in a neuron's own synaptic strengths in order to attempt to maintain an approximately invariant output PDF, nevertheless, a neuron's dynamics will be determined by its operating point parameters Θ and Γ, which are set irrespective of input statistics. Although there can be interesting interactions between synaptic plasticity and intrinsic plasticity as a neuron traverses the θ-γ plane, particularly when intrinsic plasticity is much slower than synaptic plasticity, in both models intrinsic plasticity plays second fiddle to the underlying synaptic strength fixed point structure induced in the θ-γ plane by Hebbian Appendix A Analysis of ∆ X (Γ, Θ) for…”

“…In Section 3 we consider the performance of a fixed rather than an adapting nonlinearity. We consider an extension of our earlier model of adaptation to input statistics (Elliott et al, 2008) by considering not only adaptation to changing input statistics but also adaptation to changes in synaptic strengths, in order to maintain an approximately invariant output firing rate PDF. For centred, statistically-independent and orthogonally-mixed sources with synaptic strengths normalised on the unit hypersphere, however, the mean and variance of the (standard, linearly summed) total input to a neuron are in fact independent of the neuron's synaptic strengths and thus a neuron's threshold and gain need not be changed in response to changes in synaptic strengths with such inputs.…”