Echo State Queueing Network: A new reservoir computing learning tool

Abstract

In the last decade, a new computational paradigm was introduced in the field of Machine Learning, under the name of Reservoir Computing (RC). RC models are neural networks which a recurrent part (the reservoir) that does not participate in the learning process, and the rest of the system where no recurrence (no neural circuit) occurs. This approach has grown rapidly due to its success in solving learning tasks and other computational applications. Some success was also observed with another recently proposed neural network designed using Queueing Theory, the Random Neural Network (RandNN). Both approaches have good properties and identified drawbacks.In this paper, we propose a new RC model called Echo State Queueing Network (ESQN), where we use ideas coming from RandNNs for the design of the reservoir. ESQNs consist in ESNs where the reservoir has a new dynamics inspired by recurrent RandNNs. The paper positions ESQNs in the global Machine Learning area, and provides examples of their use and performances. We show on largely used benchmarks that ESQNs are very accurate tools, and we illustrate how they compare with standard ESNs.

An Experimental Analysis of Reservoir Parameters of the Echo State Queueing Network Model

“…Recently a new RC model called the Echo State Queueing Network (ESQN) was introduced [2]. In this model the reservoir dynamics are inspired from the behavior in steady-state of a queueing network type called Generalized Queueing Network (G-Network).…”

“…Recently, a new RC model which includes ideas coming from Queueing Theory was presented under the name of Echo State Queueing Networks [2]. The model has been proven to be efficient in temporal learning tasks (for instance: Internet Traffic prediction problems).…”

“…In particular, we analyze the impact of the spectral radius and the sparsity of the reservoir on the system stability behavior as well as on the model accuracy. Another important reservoir parameter is the reservoir size which was analyzed in [2]. In our experiments, we use three benchmarks widely analyzed in the RC literature.…”

Echo State Queueing Networks: A Combination of Reservoir Computing and Random Neural Networks

“…This article is an extended version of our previous work in [4]. Observe that in the Machine Learning literature, the acronym RNN is often used for Recurrent Neural Network, that is, a neural network with circuits.…”

“…The state of the reservoir neurons is defined as in ESNs, adding a discrete clock t and making the state at t depend on what happened at t − 1. Remember that in Random Neural Networks, the neuron's loads are computed using Expressions (3), (4). The state of the ESQN reservoir is given by the following dynamics:…”