Policy iteration optimal tracking control for chaotic systems by using an adaptive dynamic programming approach

Wei, Qinglai; Liu, Derong; Xu, Yujie

doi:10.1088/1674-1056/24/3/030502

Cited by 11 publications

(5 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The iterative value functions and control laws are obtained iteratively, and the iterative control laws must stabilize the system. [7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24] An initial stabilizing control law is required, however, it is often difficult to obtain. While in most applications, fewer iterations are required, and computationally demanding is more than that of the VI iteration algorithm.…”

Section: Introductionmentioning

confidence: 99%

A novel stable value iteration-based approximate dynamic programming algorithm for discrete-time nonlinear systems

Qu¹,

Lin

2018

Chinese Phys. B

View full text Add to dashboard Cite

The convergence and stability of a value-iteration-based adaptive dynamic programming (ADP) algorithm are considered for discrete-time nonlinear systems accompanied by a discounted quadric performance index. More importantly than sufficing to achieve a good approximate structure, the iterative feedback control law must guarantee the closed-loop stability. Specifically, it is firstly proved that the iterative value function sequence will precisely converge to the optimum. Secondly, the necessary and sufficient condition of the optimal value function serving as a Lyapunov function is investigated. We prove that for the case of infinite horizon, there exists a finite horizon length of which the iterative feedback control law will provide stability, and this increases the practicability of the proposed value iteration algorithm. Neural networks (NNs) are employed to approximate the value functions and the optimal feedback control laws, and the approach allows the implementation of the algorithm without knowing the internal dynamics of the system. Finally, a simulation example is employed to demonstrate the effectiveness of the developed optimal control method.

show abstract

Section: Introductionmentioning

confidence: 99%

A novel stable value iteration-based approximate dynamic programming algorithm for discrete-time nonlinear systems

Qu¹,

Lin

2018

Chinese Phys. B

View full text Add to dashboard Cite

show abstract

“…In addition to the above methods, some researches applied optimization based methods to direct chaos to targeted regions. From the viewpoint of optimization, control of chaotic systems could be formulated as multi-modal constrained numerical optimization problems [47][48][49]. Genetic algorithm [50], simplex-annealing strategy [51], Particle swarm optimization [52], and Differential Evolution [53] have been considered.…”

Section: Introductionmentioning

confidence: 99%

Optimal targeting of nonlinear chaotic systems using a novel evolutionary computing strategy

Wang¹,

Lyu

et al. 2016

Knowledge-Based Systems

View full text Add to dashboard Cite

Control of chaotic systems to given targets is a subject of substantial and well-developed research issue in nonlinear science, which can be formulated as a class of multi-modal constrained numerical optimization problem with multi-dimensional decision variables. This investigation elucidates the feasibility of applying a novel population-based metaheuristics labelled here as Teaching-learning-based optimization to direct the orbits of discrete chaotic dynamical systems towards the desired target region. Several consecutive control steps of small bounded perturbations are made in the Teaching-learning-based optimization strategy to direct the chaotic series towards the optimal neighborhood of the desired target rapidly, where a conventional controller is effective for chaos control. Working with the dynamics of the well-known Hénon as well as Ushio discrete chaotic systems, we assess the effectiveness and efficiency of the Teaching-learning-based optimization based optimal control technique, meanwhile the impacts of the core parameters on performances are also discussed.Furthermore, possible engineering applications of directing chaotic orbits are discussed. Keywords:Chaos control; chaotic dynamics; optimization; computational intelligence;Teaching-learning-based optimization Highlights► Control of chaotic systems is formulated as constrained optimization problem.► Teaching-learning-based optimization is to direct the chaotic series.► The efficacy of TLBO based optimal control technique have been demonstrated.

show abstract

“…According to different iteration procedures, iterative ADP algorithms are classified into policy iteration and value iteration [29], respectively. In policy iteration algorithms, an admissible control law is necessary to initialize the algorithms [30][31][32]. Policy iteration algorithms for optimal control of continuous-time systems were given in [33,34].…”

Section: Introductionmentioning

confidence: 99%

A novel policy iteration based deterministic Q-learning for discrete-time nonlinear systems

Wei

Liu

2015

Sci. China Inf. Sci.

View full text Add to dashboard Cite

In this paper, a novel iterative Q-learning algorithm, called "policy iteration based deterministic Qlearning algorithm", is developed to solve the optimal control problems for discrete-time deterministic nonlinear systems. The idea is to use an iterative adaptive dynamic programming (ADP) technique to construct the iterative control law which optimizes the iterative Q function. When the optimal Q function is obtained, the optimal control law can be achieved by directly minimizing the optimal Q function, where the mathematical model of the system is not necessary. Convergence property is analyzed to show that the iterative Q function is monotonically non-increasing and converges to the solution of the optimality equation. It is also proven that any of the iterative control laws is a stable control law. Neural networks are employed to implement the policy iteration based deterministic Q-learning algorithm, by approximating the iterative Q function and the iterative control law, respectively. Finally, two simulation examples are presented to illustrate the performance of the developed algorithm.

show abstract

Policy iteration optimal tracking control for chaotic systems by using an adaptive dynamic programming approach

Cited by 11 publications

References 50 publications

A novel stable value iteration-based approximate dynamic programming algorithm for discrete-time nonlinear systems

A novel stable value iteration-based approximate dynamic programming algorithm for discrete-time nonlinear systems

Optimal targeting of nonlinear chaotic systems using a novel evolutionary computing strategy

A novel policy iteration based deterministic Q-learning for discrete-time nonlinear systems

Contact Info

Product

Resources

About