A Q-learning predictive control scheme with guaranteed stability

Beckenbach, Lukas; Osinenko, Pavel; Streif, Stefan

doi:10.1016/j.ejcon.2020.03.001

Cited by 11 publications

(9 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…II-B, utilizing offline computed local approximations V i to V ∞ in (5) requires ingredients such as, e. g., a local controller and its region of attraction or r−differentiability of system, controller and V ∞ . The former may be difficult in case the linearized dynamics around the 3 A function β : R ≥0 × R ≥0 → R ≥0 belongs to class KL if for fixed s, β(r, s) is of class K and for fixed r, β(r, s) is decreasing w. r. t. s and lims→∞ = 0.…”

Section: Resultsmentioning

confidence: 99%

“…The work [17] aims at IH optimality and uses MPC with terminal costs that involve a local Lyapunov function as well as a time-invariant weight on the terminal predicted state away from the vicinity of the origin. Other works are e. g. [6] or [3] in which the stage cost is modified using tools such as homogeneous approximations or reinforcement learning techniques, respectively. In [2], the stage cost has been designed as a monotonically increasing penality over the prediction horizon.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On performance bound estimation in NMPC with time-varying terminal cost

Beckenbach¹,

Streif²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Model predictive control (MPC) schemes are commonly designed with fixed, i. e., time-invariant, horizon length and cost functions. If no stabilizing terminal ingredients are used, stability can be guaranteed via a sufficiently long horizon. A suboptimality index can be derived that gives bounds on the performance of the MPC law over an infinite-horizon (IH). While for time-invariant schemes such index can be computed offline, less attention has been paid to time-varying strategies with adapting cost function which can be found, e. g., in learningbased optimal control. This work addresses the performance bounds of nonlinear MPC with stabilizing horizon and timevarying terminal cost. A scheme is proposed that uses the decay of the optimal finite-horizon cost and convolutes a history stack to predict the bounds on the IH performance. Based on online information on the decay rate, the performance bound estimate is improved while the terminal cost is adapted using methods from adaptive dynamic programming. The adaptation of the terminal cost leads to performance improvement over a timeinvariant scheme with the same horizon length. The approach is demonstrated in a case study.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

On performance bound estimation in NMPC with time-varying terminal cost

Beckenbach¹,

Streif²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Such a framework is especially useful for approaches where the controller needs to be automatically adjusted online, as in an adaptive control approach. In particular, an interesting application of CEGISframework arises in the stability analysis of adaptive dynamic programming schemes, where one has to guarantee that the desired critic parameters force the positive definiteness of the so-called Q-function approximations at each time step of the scheme [32], [33]. Using CEGIS one may construct the required Q-functions and verify the conditions of the LaSalle-Yoshizawa theorem [34,Theorem 18] simultaneously.…”

Section: Discussionmentioning

confidence: 99%

Synthesis of Lyapunov Functions using Formal Verification

Lukas¹,

Devadze²,

Streif³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Recent employments of SMT solvers within the Lyapunov function synthesis provided effective tools for automated construction of Lyapunov functions alongside with sound computer-assisted certificates. The main benefit of the suggested approach is the formal correctness and elimination of the numerical uncertainty. In the present work we extend the SMTbased synthesis approach for wider classes of continuous and discrete-time systems. Additionally, we address constructions of Lyapunov functions for state-dependent switching systems. We illustrate our approach by means of various examples from the control systems literature.

show abstract

“…Zeng et al, [23] aimed to increase the control performance of a selective catalytic reduction (SCR) denitrification system through modeling and disturbance rejection. Beckenbach et al, [24] analyzed model-based predictive controllers used to manage control tasks with constraints on the state. Bünning et al, [25] developed a model predictive control method for room temperature control in buildings.…”

Section: A Recent Advancements In Crane Spreader and Cargo Stabilizationmentioning

confidence: 99%

Application of Neural Network Predictive Control Methods to Solve the Shipping Container Sway Control Problem in Quay Cranes

2021

View full text Add to dashboard Cite

Smart control systems are mostly applied in industry to control the movements of heavy machinery while optimizing overall operational efficiency. Major shipping companies use large quay cranes to load and unload containers from ships and still rely on the experience of on-site operators to perform transportation control procedures using joysticks and visual contact methods. This paper presents the research results of an EU-funded project for the Klaipeda container terminal to develop a novel container transportation security and cargo safety assurance method and system. It was concluded that many risks arise during the container handling procedures performed by the quay cranes and operators. To minimize these risks, the authors proposed controlling the sway of the spreader using a model predictive control method which applies a multi-layer perceptron (MLP) neural network (NN). The paper analyzes current neural network architectures and case studies and provides the engineering community with a unique case study which applies real operation statistical data. Several key training algorithms were tested, and the initial results suggest that the Levenberg-Marquardt (LM) algorithm and variable learning rate backpropagation perform better than methods which use the multi-layer perceptron neural network structure.

show abstract

A Q-learning predictive control scheme with guaranteed stability

Cited by 11 publications

References 52 publications

On performance bound estimation in NMPC with time-varying terminal cost

On performance bound estimation in NMPC with time-varying terminal cost

Synthesis of Lyapunov Functions using Formal Verification

Application of Neural Network Predictive Control Methods to Solve the Shipping Container Sway Control Problem in Quay Cranes

Contact Info

Product

Resources

About