Coherent Infomax as a Computational Goal for Neural Systems

Abstract

Signal processing in the cerebral cortex is thought to involve a common multi-purpose algorithm embodied in a canonical cortical micro-circuit that is replicated many times over both within and across cortical regions. Operation of this algorithm produces widely distributed but coherent and relevant patterns of activity. The theory of Coherent Infomax provides a formal specification of the objectives of such an algorithm. It also formally derives specifications for both the short-term processing dynamics and for the learning rules whereby the connection strengths between units in the network can be adapted to the environment in which the system finds itself. A central assumption of the theory is that the local processors can combine reliable signal coding with flexible use of those codes because they have two classes of synaptic connection: driving connections which specify the information content of the neural signals, and contextual connections which modulate that signal processing. Here, we make the biological relevance of this theory more explicit by putting more emphasis upon the contextual guidance of ongoing processing, by showing that Coherent Infomax is consistent with a particular Bayesian interpretation for the contextual guidance of learning and processing, by explicitly specifying rules for on-line learning, and by suggesting approximations by which the learning rules can be made computationally feasible within systems composed of very many local processors.

On the functions, mechanisms, and malfunctions of intracortical contextual modulation

“…The second, discussed in Section 6.2, is the theory of 'coherent infomax' (e.g. Kay and Phillips, 2011). Both use information-theoretic notions to formulate hypotheses concerning the long-term objectives of neural systems.…”

“…Learning rules for modifying the strengths of their synaptic connections were derived analytically from the formally-specified objective, and shown to have biological plausibility. It was shown that contextual modulation was required to achieve the objective (Smyth, Phillips and Kay, 1996), and that the learning rules can be applied exactly in simple cases (Kay, Floreano and Phillips, 1998) and by approximation in more complex cases (Kay and Phillips, 2011). The dynamics of systems built from such local neural processing elements were shown to depend in functionally useful ways upon predictive relationships between the variables defined on their driving inputs.…”

“…The problem is that the theory requires the modulatory and driving inputs to come from different sources and to be integrated separately prior to their interaction (Kay and Phillips, 2011). NMDARs are co-localized with AMPA receptors, however, so they cannot receive inputs from sources that differ from AMPA receptors, and they cannot first be integrated separately before being used to modulate response to the integrated RF input.…”

“…The modulatory inputs must be integrated separately before being used to modulate response to the integrated driving inputs (Kay and Phillips, 2011).The two separate sites of integration shown in Figure 2A could be implemented by two distinct sets of neurons. Section 3 will cite evidence that local cortical circuits contain neurons that integrate inputs from various contextual sources before using their outputs to either amplify or suppress the responses of other cells in the local circuit.…”

“…That is done by the theory of coherent infomax, which proposes a general account of cortical function in which contextual modulation plays a central role that is crucial to both learning and processing (Phillips, Kay and Smyth, 1995;Kay, Floreano and Phillips, 1998;Kay and Phillips, 2011). Contextual modulation as conceived within that theory is therefore examined further here.…”