Computational aspects of feedback in neural circuits

Abstract

It has previously been shown that generic cortical microcircuit models can perform complex real-time computations on continuous input streams, provided that these computations can be carried out with a rapidly fading memory. We investigate the computational capability of such circuits in the more realistic case where not only readout neurons, but in addition a few neurons within the circuit, have been trained for specific tasks. This is essentially equivalent to the case where the output of trained readout neurons is fed back into the circuit. We show that this new model overcomes the limitation of a rapidly fading memory. In fact, we prove that in the idealized case without noise it can carry out any conceivable digital or analog computation on time-varying inputs. But even with noise, the resulting computational model can perform a large class of biologically relevant real-time computations that require a nonfading memory. We demonstrate these computational implications of feedback both theoretically, and through computer simulations of detailed cortical microcircuit models that are subject to noise and have complex inherent dynamics. We show that the application of simple learning procedures (such as linear regression or perceptron learning) to a few neurons enables such circuits to represent time over behaviorally relevant long time spans, to integrate evidence from incoming spike trains over longer periods of time, and to process new information contained in such spike trains in diverse ways according to the current internal state of the circuit. In particular we show that such generic cortical microcircuits with feedback provide a new model for working memory that is consistent with a large set of biological constraints. Although this article examines primarily the computational role of feedback in circuits of neurons, the mathematical principles on which its analysis is based apply to a variety of dynamical systems. Hence they may also throw new light on the computational role of feedback in other complex biological dynamical systems, such as, for example, genetic regulatory networks.

What artificial grammar learning reveals about the neurobiology of syntax

“…Moreover, under the additional assumption that the available processing time is finite, the same conclusion follows for the continuous-time case, if finite temporal precision or temporal noise is assumed. This is essentially what the technical results of Maass and colleagues (Maass & Orponen, 1997Maass & Sontag, 1999;Maass et al, 2007), as well as others (Casey, 1996;Siegelmann, 1999), entail.…”

“…In passing, we note that a qualitative match between the performance of simple recurrent networks and human comprehension of nested (context-free) and crossed (context-sensitive) dependencies has been reported (Christiansen & Chater, 1999;Christiansen & MacDonald, 2009). Because (in a technical sense), noisy or discrete simple recurrent networks are finite-state architectures (Casey, 1996;Maass, Joshi, & Sontag, 2007;Maass & Orponen, 1998;Maass & Sontag, 1999;see also, Petersson, 2005b;Petersson, Grenholm, & Forkstam, 2005), these results suggest that actual language processing uses no more on-line memory resources than can be provided by a finite-state architecture. These simulations, of course, only illustrate that recurrent networks can handle (bounded) non-regular processing at some level of proficiency.…”

The role of feedback in morphological computation with compliant bodies

“…We present a theoretical framework for morphological computation, which is based on a result by Maass et al (2007). They proved that a certain class of nonlinear dynamical systems (which can have the property of fading memory) gain computational power to emulate arbitrary nonlinear systems (which can have persistent memory), by adding simply a suitable static (memoryless) feedback and a suitable static (memoryless) readout function.…”

Time scales of memory, learning, and plasticity

“…The abstract idea of this concept is that a complex network of calculating identities (e.g., neurons) is so diverse that each task is solved somewhere within the network (Maass et al 2002;Buonomano and Maass 2009;Maass 2010). However, one problem with this approach is the capacity, which depends sublinearly on the number of neurons (Ganguli et al 2008); another problem is the read-out of the task-specific information from the network (Maass et al 2007;Legenstein et al 2008). …”