Near Optimal Output-Feedback Control of Nonlinear Discrete-time Systems in Nonstrict Feedback Form with Application to Engines

Shih, Peter; Kaul, Brian; Jagannathan, S.; Drallmeier, J. A.

doi:10.1109/ijcnn.2007.4370989

Cited by 8 publications

(3 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus in this paper, as an application scenario, the controller with UUB guarantee is learned to solve control tasks with safety constraints. In terms of learning a controller, UUB is well known in the context of adaptive dynamic programming (ADP) on (1) UUB of the states or tracking error in uncertain systems [9][10][11] and (2) UUB of the estimation error of critic function and controller [9,12,13]. However, in ADP, the model structure, not necessarily the model parameters, should be known a prior.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement Learning Control of Constrained Dynamic Systems with Uniformly Ultimate Boundedness Stability Guarantee

Han

Tian

Zhang

et al. 2020

Preprint

View full text Add to dashboard Cite

Reinforcement learning (RL) is promising for complicated stochastic nonlinear control problems. Without using a mathematical model, an optimal controller can be learned from data evaluated by certain performance criteria through trial-and-error. However, the data-based learning approach is notorious for not guaranteeing stability, which is the most fundamental property for any control system. In this paper, the classic Lyapunov's method is explored to analyze the uniformly ultimate boundedness stability (UUB) solely based on data without using a mathematical model. It is further shown how RL with UUB guarantee can be applied to control dynamic systems with safety constraints. Based on the theoretical results, both off-policy and on-policy learning algorithms are proposed respectively. As a result, optimal controllers can be learned to guarantee UUB of the closed-loop system both at convergence and during learning. The proposed algorithms are evaluated on a series of robotic continuous control tasks with safety constraints. In comparison with the existing RL algorithms, the proposed method can achieve superior performance in terms of maintaining safety. As a qualitative evaluation of stability, our method shows impressive resilience even in the presence of external disturbances.

show abstract

Section: Introductionmentioning

confidence: 99%

Reinforcement Learning Control of Constrained Dynamic Systems with Uniformly Ultimate Boundedness Stability Guarantee

Han

Tian

Zhang

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Liu et al promoted an ADP approach and presented an engine torque and exhaust air–fuel ratio controller using an adaptive critic algorithm . Shih extended stability proofs of nonlinear discrete‐time systems in nonstrict feedback form and applied it to exhaust gas recirculation control of a spark engine upon the basis of reinforcement learning dual control . Lu et al applied a direct HDP to a large power system damping oscillations control problem .…”

Section: Introductionmentioning

confidence: 99%

“…As for industrial applications, ADP has focused on flight control , engine control , power systems , auto‐landing problem , missile systems , artificial intelligence , etc. Enns and Si demonstrated a model‐free nonlinear multi‐input multi‐output (MIMO) helicopter flight controller .…”

Section: Introductionmentioning

confidence: 99%

An approximate dynamic programming method for multi‐input multi‐output nonlinear system

Huang

2011

Optim Control Appl Methods

View full text Add to dashboard Cite

SUMMARYApproximate dynamic programming is a useful tool in solving multi-stage decision optimal control problems. In this work, we first promote the action-dependent heuristic dynamic programming method to multi-input multi-output control system by extending its action network to multi-output form. The detailed derivation is also given. We then apply this method to the fluctuation control of a spark engine idle speed. An engine idling model is set up to verify the control effect of this method. Results here show that this method requires several iterations to suppress unbalanced combustion by manipulating spark ignition timing. This method provides an alternative for a simpler multi-input multi-output approximate dynamic programming scheme. Moreover, it has a faster iteration convergence effect. The derivation of this method also has a rigorous mathematical basis. Although illustrated for engines, this control system framework should also be applicable to general multi-input multi-output nonlinear system.

show abstract

Overview

Zhang

Liu

Luo

et al. 2013

Communications and Control Engineering

View full text Add to dashboard Cite

Near Optimal Output-Feedback Control of Nonlinear Discrete-time Systems in Nonstrict Feedback Form with Application to Engines

Cited by 8 publications

References 6 publications

Reinforcement Learning Control of Constrained Dynamic Systems with Uniformly Ultimate Boundedness Stability Guarantee

Reinforcement Learning Control of Constrained Dynamic Systems with Uniformly Ultimate Boundedness Stability Guarantee

An approximate dynamic programming method for multi‐input multi‐output nonlinear system

Overview

Contact Info

Product

Resources

About