Is there a Liquid State Machine in the Bacterium Escherichia Coli?

Jones, Ben; Stekel, Dov J.; Rowe, Jonathan E.; Fernando, Chrisantha

doi:10.1109/alife.2007.367795

Cited by 52 publications

(40 citation statements)

References 12 publications

(13 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many other forms of reservoirs can be found in the literature (e.g. (Jones et al, 2007;Deng & Zhang, 2007;Dockendorf et al, 2009;Bush & Anderson, 2005;Ishii et al, 2004;Schmidhuber et al, 2007;Ajdari Rad et al, 2008)). However, exactly what aspects of reservoirs are responsible for their often reported superior modelling capabilities (Jaeger, 2001(Jaeger, , 2002aJaeger & Hass, 2004;Maass et al, 2004;Tong et al, 2007) is still unclear.…”

Section: Resultsmentioning

confidence: 99%

Simple Deterministically Constructed Cycle Reservoirs with Regular Jumps

Rodan

Tiňo

2012

Neural Computation

View full text Add to dashboard Cite

A new class of state-space models, reservoir models, with a fixed state transition structure (the "reservoir") and an adaptable readout from the state space has recently emerged as a way for time series processing/modelling. Echo State Network (ESN) is one of the simplest, yet powerful, reservoir models. ESN models are generally constructed in a randomized manner. In our previous study (Rodan & Tino, 2011) we showed that a very simple, cyclic, deterministically generated reservoirs can yield performance competitive with standard ESN. In this contribution we extend our previous study in three aspects: 1) We introduce a novel simple deterministic reservoir model, Cycle Reservoir with Jumps (CRJ), with highly constrained weight values, that has superior performance to standard ESN on a variety of temporal tasks of different origin and characteristics. 2) We elaborate on the possible link between reservoir characterizations, such as eigenvalue distribution of the reservoir matrix or pseudo-Lyapunov exponent of the input-driven reservoir dynamics, and the model performance. It has been suggested that a uniform coverage of the unit disk by such eigenvalues can lead to superior model performances. We show that despite highly constrained eigenvalue distribution, CRJ consistently outperform ESN (that have much more uniform eigenvalue coverage of the unit disk). Also, unlike in the case of ESN, pseudo-Lyapunov exponents of the selected 'optimal' CRJ models are consistently negative. 3) We present a new framework for determining short term memory capacity of linear reservoir models to a high degree of precision. Using the framework we study the effect of shortcut connections in the CRJ reservoir topology on its memory capacity.

show abstract

Section: Resultsmentioning

confidence: 99%

Simple Deterministically Constructed Cycle Reservoirs with Regular Jumps

Rodan

Tiňo

2012

Neural Computation

View full text Add to dashboard Cite

show abstract

“…Originally, this idea had been separately developed for sigmoid nodes (Jaeger, 2001) and spiking neurons (Maass et al, 2002), where the respective approaches were called Echo State Networks (ESN) and Liquid State Machines (LSM), but there are no limitations to the types of networks one could use (Fernando and Sojakka, 2003;Jones et al, 2007). Indeed, the network does not even have to be a true neural network in the common sense: any nonlinear, dynamical system with the right properties (most importantly: 'fading memory' (Jaeger, 2001)) can potentially be used in this approach.…”

Section: Introductionmentioning

confidence: 99%

Recurrent Kernel Machines: Computing with Infinite Echo State Networks

Hermans

Schrauwen

2012

Neural Computation

View full text Add to dashboard Cite

Echo State Networks are large, random recurrent neural networks with a single trained linear readout layer. Despite the untrained nature of the recurrent weights, they are capable of performing universal computations on temporal input data, which makes them interesting for both theoretical research and practical applications. The key to their success lies in the fact that the network computes a broad set of nonlinear, spatiotemporal mappings of the input data, on which linear regression or classification can easily be performed. One could consider the reservoir as a spatiotemporal kernel, in which the mapping to a high-dimensional space is computed explicitly. In this paper, we build on this idea and extend the concept of ESNs to infinite sized recurrent neural networks, which can be considered as recursive kernels that subsequently can be used to create recursive SVMs. We present the theoretical framework, provide several practical examples of recursive kernels, and apply them to typical temporal tasks.

show abstract

“…The popularity of RC stems from its ease of use, combined with its computational capabilities that match or exceed the state-of-the-art for a broad range of applications such as speech recognition, time series prediction, pattern classification and robotics 1,[7][8][9][10] . Its lenient requirements for the reservoir have led to implementations on several hardware platforms ranging from a basin of water to cellular neural networks and bacteria [11][12][13] .…”

mentioning

confidence: 99%

Experimental demonstration of reservoir computing on a silicon photonics chip

et al. 2014

View full text Add to dashboard Cite

In today's age, companies employ machine learning to extract information from large quantities of data. One of those techniques, reservoir computing (RC), is a decade old and has achieved state-of-the-art performance for processing sequential data. Dedicated hardware realizations of RC could enable speed gains and power savings. Here we propose the first integrated passive silicon photonics reservoir. We demonstrate experimentally and through simulations that, thanks to the RC paradigm, this generic chip can be used to perform arbitrary Boolean logic operations with memory as well as 5-bit header recognition up to 12.5 Gbit s À 1 , without power consumption in the reservoir. It can also perform isolated spoken digit recognition. Our realization exploits optical phase for computing. It is scalable to larger networks and much higher bitrates, up to speeds 4100 Gbit s À 1 . These results pave the way for the application of integrated photonic RC for a wide range of applications.

show abstract

Is there a Liquid State Machine in the Bacterium Escherichia Coli?

Cited by 52 publications

References 12 publications

Simple Deterministically Constructed Cycle Reservoirs with Regular Jumps

Simple Deterministically Constructed Cycle Reservoirs with Regular Jumps

Recurrent Kernel Machines: Computing with Infinite Echo State Networks

Experimental demonstration of reservoir computing on a silicon photonics chip

Contact Info

Product

Resources

About