Prediction of telephone calls load using Echo State Network with exogenous variables

Bianchi, Filippo Maria; Scardapane, Simone; Uncini, Aurelio; Rizzi, Antonello; Sadeghian, Alireza

doi:10.1016/j.neunet.2015.08.010

Cited by 78 publications

(60 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…First, we compare λ with L max , the value of the longest diagonal line in an RP-see (10) and related discussion. L max is a global indicator of stability that we show here to be highly correlated with λ. Successively, we show that the edge of stability determined according to (15) identifies more accurately the region of the hyperparameter space where γ is maximized. To produce more interpretable results, for both input signals, we map independently the values of each RQA measure, obtained for the different values of ρ and ω i , in the [0, 1] interval by using a unity-based normalization.…”

Section: B Characterization Of Reservoirs With Rqamentioning

confidence: 89%

“…This is justified by the fact that high-amplitude signals tend to saturate nonlinear activation functions and cause the poles to shrink toward the origin. This results in a system with (15). For the sake of readability, here we show only three RQA measures; see Table II for detailed results.…”

Section: B Characterization Of Reservoirs With Rqamentioning

confidence: 99%

“…12(c)] and the RQA measures start to fluctuate for much higher values of ρ, close to 1.5. (15). For the sake of readability, here we show only three RQA measures; see Table II for detailed results.…”

Section: Mg Time Seriesmentioning

confidence: 99%

See 2 more Smart Citations

Investigating Echo-State Networks Dynamics by Means of Recurrence Analysis

Bianchi

Livi

Alippi

2018

IEEE Trans. Neural Netw. Learning Syst.

Self Cite

View full text Add to dashboard Cite

Abstract-In this paper, we elaborate over the well-known interpretability issue in echo-state networks (ESNs). The idea is to investigate the dynamics of reservoir neurons with timeseries analysis techniques developed in complex systems research. Notably, we analyze time series of neuron activations with recurrence plots (RPs) and recurrence quantification analysis (RQA), which permit to visualize and characterize highdimensional dynamical systems. We show that this approach is useful in a number of ways. First, the 2-D representation offered by RPs provides a visualization of the high-dimensional reservoir dynamics. Our results suggest that, if the network is stable, reservoir and input generate similar line patterns in the respective RPs. Conversely, as the ESN becomes unstable, the patterns in the RP of the reservoir change. As a second result, we show that an RQA measure, called L max , is highly correlated with the well-established maximal local Lyapunov exponent. This suggests that complexity measures based on RP diagonal lines distribution can quantify network stability. Finally, our analysis shows that all RQA measures fluctuate on the proximity of the so-called edge of stability, where an ESN typically achieves maximum computational capability. We leverage on this property to determine the edge of stability and show that our criterion is more accurate than two well-known counterparts, both based on the Jacobian matrix of the reservoir. Therefore, we claim that RPs and RQA-based analyses are valuable tools to design an ESN, given a specific problem.Index Terms-Dynamics, echo-state network (ESN), recurrence plot (RP), recurrence quantification analysis (RQA), nonlinear time series analysis.

show abstract

Section: B Characterization Of Reservoirs With Rqamentioning

confidence: 89%

Section: B Characterization Of Reservoirs With Rqamentioning

confidence: 99%

See 1 more Smart Citation

Investigating Echo-State Networks Dynamics by Means of Recurrence Analysis

Bianchi

Livi

Alippi

2018

IEEE Trans. Neural Netw. Learning Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Then, a feedforward output layer, called readout, is trained on top of the reservoir, using standard techniques from linear approximation theory, most notably ridge regression. Despite this simplification, ESNs have obtained remarkable results in many fields, including medical image segmentation [21], load prediction [22], language generation [23], and several others. Based on the strict separation between reservoir and readout, in [11] we proposed a distributed algorithm for training ESNs, by using the well-known optimization routine known as alternating direction method of multipliers (ADMM) [24].…”

Section: Introductionmentioning

confidence: 99%

“…At the same time, standard ridge regression may not be the most suitable training algorithm for ESNs, as demonstrated by works exploring readouts trained via support vector based algorithms [27], elastic net penalties [22], and others. Specifically, in this paper we are concerned with training a sparse readout, i.e.…”

Section: Introductionmentioning

confidence: 99%

Distributed Reservoir Computing with Sparse Readouts [Research Frontier]

Scardapane

Panella

Comminiello

et al. 2016

IEEE Comput. Intell. Mag.

Self Cite

View full text Add to dashboard Cite

In a network of agents, a widespread problem is the need to estimate a common underlying function starting from locally distributed measurements. Real-world scenarios may not allow the presence of centralized fusion centers, requiring the development of distributed, message-passing implementations of the standard machine learning training algorithms. In this paper, we are concerned with the distributed training of a particular class of recurrent neural networks, namely echo state networks (ESNs). In the centralized case, ESNs have received considerable attention, due to the fact that they can be trained with standard linear regression routines. Based on this observation, in our previous work we have introduced a decentralized algorithm, framed in the distributed optimization field, in order to train an ESN. In this paper, we focus on an additional sparsity property of the output layer of ESNs, allowing for very efficient implementations of the resulting networks. In order to evaluate the proposed algorithm, we test it on two well-known prediction benchmarks, namely the Mackey-Glass chaotic time series and the 10th order nonlinear auto regressive moving average (NARMA) system.

show abstract

Randomness in neural networks: an overview

Scardapane

Wang

2017

WIREs Data Min & Knowl

Self Cite

231

144

View full text Add to dashboard Cite

Neural networks, as powerful tools for data mining and knowledge engineering, can learn from data to build feature-based classifiers and nonlinear predictive models. Training neural networks involves the optimization of nonconvex objective functions, and usually, the learning process is costly and infeasible for applications associated with data streams. A possible, albeit counterintuitive, alternative is to randomly assign a subset of the networks' weights so that the resulting optimization task can be formulated as a linear least-squares problem. This methodology can be applied to both feedforward and recurrent networks, and similar techniques can be used to approximate kernel functions. Many experimental results indicate that such randomized models can reach sound performance compared to fully adaptable ones, with a number of favorable benefits, including (1) simplicity of implementation, (2) faster learning with less intervention from human beings, and (3) possibility of leveraging overall linear regression and classification algorithms (e.g., ℓ 1 norm minimization for obtaining sparse formulations). This class of neural networks attractive and valuable to the data mining community, particularly for handling large scale data mining in real-time. However, the literature in the field is extremely vast and fragmented, with many results being reintroduced multiple times under different names. This overview aims to provide a self-contained, uniform introduction to the different ways in which randomization can be applied to the design of neural networks and kernel functions. A clear exposition of the basic framework underlying all these approaches helps to clarify innovative lines of research, open problems, and most importantly, foster the exchanges of well-known results throughout different communities.

show abstract

Prediction of telephone calls load using Echo State Network with exogenous variables

Cited by 78 publications

References 46 publications

Investigating Echo-State Networks Dynamics by Means of Recurrence Analysis

Investigating Echo-State Networks Dynamics by Means of Recurrence Analysis

Distributed Reservoir Computing with Sparse Readouts [Research Frontier]

Randomness in neural networks: an overview

Contact Info

Product

Resources

About