A new self-learning optimal control laws for a class of discrete-time nonlinear systems based on ESN architecture

Song, Ruizhuo; Xiao, Wendong; Sun, Changyin

doi:10.1007/s11432-013-4954-y

Cited by 21 publications

(8 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dynamic Reservoir contains a large number of sparsely connected neurons, which mimics the working principle of the human brain neurons. They accept information from the input layer just as the human brain neurons receive stimuli from the outside world [14].…”

Section: Basic Esnmentioning

confidence: 99%

Improved Echo State Network (ESN) for the Prediction of Network Traffic

Ye¹,

Lv²,

Jiang

et al. 2018

Proceedings of the 11th EAI International Conference on Mobile Multimedia Communications

View full text Add to dashboard Cite

With the development of computer and network technologies, the network traffic has come to an explosive growth. Considering the non-linearity of network data, we choose echo state network (ESN) to predict the data. ESN is a novel kind of recurrent neural networks, where a reservoir is generated randomly and only output weight matrix is adaptable. In our paper, we propose an improved network of ESN considering its weekness on selection of matrix initialization and activation function. The improved network defines the scope of matrix initialization and replaces the activation function of middle layer with wavelet function. And result shows that our improved network is more effective compared to original ESN. Our evaluation index is normalized mean square error (NMSE), and it drops from 0.7435 to 0.5852 by making improvements. network predicts well. Numerous experiments show that ESN has better adaptability to nonlinear time series.The rest of this paper is organized as follows. Section 2 describes previous related work about prediction of network traffic. In Section 3, we make an analysis of the data acquired from telecom operator. In Section 4, we introduce the structure of echo state network (ESN) and improvement of model. Finally, we give simulation results and conclusion in section 6. Related workBoth short-term and long-term prediction of network traffic are feasible directions for research. With historical traffic data and suitable method, we can have a general idea of the size of future traffic. With the prediction, operators can adjust the distribution of network resource in time. Also, equipment manufacturers can replace network devices with insufficient load capacity in advance.In recent years, there have been many technologies applied in the filed of traffic prediction. Pang et al [4] establised an adpative fuzzy traffic predicator based on the theory of fuzzy system [5] . They used nearest neighborhood clustering learning algrothim to present the fuzzy traffic predicator, and the results showed that the predicator was accurate and flexible applied in ATM networks. Sang et al [6] predicted network traffic using two stationary traffic models, including the Auto-Regressive Moving Average(ARMA) [7] model and the Markov-Modulated Possion Process(MMPP) [8] model. These two models mainly use mathematical analysis and experiments to do the predictability analysis of network traffic. Besides, wavelet analysis has been one of the most effective method to deal with non-stationary time series for traffic prediction. Wang et al [9] proposed a novel method of combining wavelet and RLS to forecast the Internet traffic and results showed this method achieved extraodinary accuracy comapred to other models. As we all know, Artificial Intelligence (AI) has been applied in many areas widely, such as expert system, fuzzy reasoning and fuzzy neural. Specially, there are many techniques about neural networks(NNs) used in the field of prediction. Neural networks are suitable for learing more complex nonlinear relationships. Mo...

show abstract

Section: Basic Esnmentioning

confidence: 99%

Improved Echo State Network (ESN) for the Prediction of Network Traffic

Ye¹,

Lv²,

Jiang

et al. 2018

Proceedings of the 11th EAI International Conference on Mobile Multimedia Communications

View full text Add to dashboard Cite

show abstract

“…Many approaches have been proposed to obtain the approximate solution of the HJB equation such as the adaptive dynamic programming (DP). This technique is classified into several schemes including the heuristic DP [10], dual heuristic DP [11], action-dependent DP, and Q-learning DP [12]. However, the DP policy is not computationally tenable to run and solves the time-varying HJB equations [13].…”

Section: Introductionmentioning

confidence: 99%

On a New and Efficient Numerical Technique to Solve a Class of Discrete-time Nonlinear Optimal Control Problems

Abadi¹,

Vaziri²,

Jajarmi³

2019

JESA

View full text Add to dashboard Cite

This article proposes a new and efficient iterative procedure to solve a class of discrete-time nonlinear optimal control problems. Based on the Pontryagin's maximum principle, the necessary optimality conditions are formulated in the form of a nonlinear discrete boundary value problem (BVP). This problem is then reduced into a sequence of linear discrete BVPs by applying a series expansion approach called the modal series method. Solving the aforementioned sequence by using the techniques of solving linear difference equations, the optimal control law is derived in the form of a uniformly convergent series. In order to demonstrate the efficiency of the proposed method in practice, an iterative algorithm with a fast rate of convergence is provided. In a recursive manner, only a few iterations are needed to find a suboptimal control law with enough accuracy. The effectiveness of this new technique is verified by solving some numerical examples.

show abstract

“…Meanwhile, for traditional ADP methods, the solution to infinite‐horizon optimal control of discrete‐time nonlinear systems is based on value iterations and policy iterations . The training requires large number of iterations.…”

Section: Introductionmentioning

confidence: 99%

Online identifier–actor–critic algorithm for optimal control of nonlinear systems

Lin

Wei

Liu

2016

Optim Control Appl Methods

View full text Add to dashboard Cite

Summary In this paper, a novel identifier–actor–critic optimal control scheme is developed for discrete‐time affine nonlinear systems with uncertainties. In contrast to traditional adaptive dynamic programming methodology, which requires at least partial knowledge of the system dynamics, a neural‐network identifier is employed to learn the unknown control coefficient matrix working together with actor–critic‐based scheme to solve the optimal control online. The critic network learns the approximate value function at each step. The actor network attempts to improve the current policy based on the approximate value function. The weights of all neural networks are updated at each sampling instant. Lyapunov theory is utilized to prove the stability of closed‐loop system. It shows that system states and neural network weights are uniformly ultimately bounded. Finally, simulations are provided to illustrate the effectiveness of the developed method. Copyright © 2016 John Wiley & Sons, Ltd.

show abstract

A new self-learning optimal control laws for a class of discrete-time nonlinear systems based on ESN architecture

Cited by 21 publications

References 35 publications

Improved Echo State Network (ESN) for the Prediction of Network Traffic

Improved Echo State Network (ESN) for the Prediction of Network Traffic

On a New and Efficient Numerical Technique to Solve a Class of Discrete-time Nonlinear Optimal Control Problems

Online identifier–actor–critic algorithm for optimal control of nonlinear systems

Contact Info

Product

Resources

About