Event-triggered H-infinity control for unknown continuous-time linear systems using Q-learning

Vamvoudakis, Kyriakos G.; Ferraz, Henrique

doi:10.1109/cdc.2016.7798458

Cited by 10 publications

(13 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [22], we have brought together intermittent mechanisms to alleviate the burden of an actor-critic framework. This was later extended for systems with unknown dynamics under a Q-learning framework in [23]. The authors of [24] developed a controller with intermittent communication for a multi-agent system, whose dwell-time conditions, needed for stability, as well as safety constraints were expressed by metric temporal logic specifications.…”

Section: Related Workmentioning

confidence: 99%

Temporal-Logic-Based Intermittent, Optimal, and Safe Continuous-Time Learning for Trajectory Tracking

Kanellopoulos,

Fotiadis,

Sun

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we develop safe reinforcementlearning-based controllers for systems tasked with accomplishing complex missions that can be expressed as linear temporal logic specifications, similar to those required by search-andrescue missions. We decompose the original mission into a sequence of tracking sub-problems under safety constraints. We impose the safety conditions by utilizing barrier functions to map the constrained optimal tracking problem in the physical space to an unconstrained one in the transformed space. Furthermore, we develop policies that intermittently update the control signal to solve the tracking sub-problems with reduced burden in the communication and computation resources. Subsequently, an actor-critic algorithm is utilized to solve the underlying Hamilton-Jacobi-Bellman equations. Finally, we support our proposed framework with stability proofs and showcase its efficacy via simulation results.

show abstract

Section: Related Workmentioning

confidence: 99%

Temporal-Logic-Based Intermittent, Optimal, and Safe Continuous-Time Learning for Trajectory Tracking

Kanellopoulos,

Fotiadis,

Sun

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Proof 1. The proof for this part can be referred to the work of Vamvoudakis and Ferraz and is omitted here. 2.…”

Section: Model‐based Intermittent Feedback Designmentioning

confidence: 99%

“…Denote the worst‐case disturbance for the intermittent feedback u s as

w_{s}^{*}

. Then, the control policy is optimal with the optimal cost with the worst‐case disturbance provided in the work of Vamvoudakis and Ferraz as follows:

J (x_{0}; u^{*} (\cdot), w^{*} (\cdot), u_{s} (\cdot)) = \frac{1}{2} x_{0}^{T} P x_{0} + \frac{1}{2} \int_{0}^{\infty} {[u^{*} (τ) - u_{s} (τ)]}^{T} R [u^{*} (τ) - u_{s} (τ)] d τ - \frac{γ^{2}}{2} \int_{0}^{\infty} {[w^{*} (τ) - w_{s}^{*} (τ)]}^{2} d τ .

Then, this shall be a trivial extension of the disturbance modeled as an impulsive system as is done with the control. Both the worst‐case disturbance

w_{d}^{*}

and the corresponding optimal continuous feedback u ∗ will be able to be learned using the intermittent Q ‐learning algorithm.…”

Section: Model‐free Intermittent Feedback Design: Q‐learning Algorithmmentioning

confidence: 99%

“…In this section, an example taken from is simulated to show the effectiveness of the proposed model‐free dynamic intermittent control policy. Consider the linear system

\overset{x}{true} = A x + B_{1} u + B_{2} w

, where

A = ([], \begin{matrix} center \\ array - 2 & array 1 \\ array 0 & array 3 \end{matrix})

B_{1} = ([], \begin{matrix} center \\ array 1 \\ array 2 \end{matrix})

and

B_{2} = ([], \begin{matrix} center \\ array 1 \\ array - 1 \end{matrix})

are unknown matrices to the controller designer The parameters for the utility function in are selected as H = 0.01 I 2 and R = 0.01.…”

Section: Simulation Studymentioning

confidence: 99%

“…In this section, an example taken from 35 are unknown matrices to the controller designer The parameters for the utility function in (5) are selected as H = 0.01I 2 and R = 0.01. Both the static and dynamic model-free co-design approaches discussed in this paper are used to develop the event-triggering condition and optimal feedback gain simultaneously.…”

Section: Simulation Studymentioning

confidence: 99%

See 2 more Smart Citations

Dynamic intermittent Q‐learning–based model‐free suboptimal co‐design of ‐stabilization

Yang

Vamvoudakis

Ferraz

et al. 2019

Intl J Robust & Nonlinear

Self Cite

View full text Add to dashboard Cite

Summary This paper proposes an intermittent model‐free learning algorithm for linear time‐invariant systems, where the control policy and transmission decisions are co‐designed simultaneously while also being subjected to worst‐case disturbances. The control policy is designed by introducing an internal dynamical system to further reduce the transmission rate and provide bandwidth flexibility in cyber‐physical systems. Moreover, a Q‐learning algorithm with two actors and a single critic structure is developed to learn the optimal parameters of a Q‐function. It is shown by using an impulsive system approach that the closed‐loop system has an asymptotically stable equilibrium and that no Zeno behavior occurs. Furthermore, a qualitative performance analysis of the model‐free dynamic intermittent framework is given and shows the degree of suboptimality concerning the optimal continuous updated controller. Finally, a numerical simulation of an unknown system is carried out to highlight the efficacy of the proposed framework.

show abstract

Sparse event‐triggered control of linear systems

Banno

Azuma

Ariizumi

et al. 2022

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

In event‐triggered control, a situation where the control input must be sparse often arises. Therefore, in this study, we propose sparse event‐triggered control, meaning that the control input is sparse and updated in an event‐triggered manner. First, we present a model‐based method for sparse event‐triggered control of linear systems, where the event condition is defined by a Lyapunov function. The resulting control input is proven to be sparse and the control system is confirmed to be asymptotically stable. Second, we extend it to a data‐driven version, where the event condition is adaptively updated from online data on the state trajectory. Finally, we discuss the possibility of extending our framework to two cases of disturbance and nonlinear dynamics.

show abstract

Event-triggered H-infinity control for unknown continuous-time linear systems using Q-learning

Cited by 10 publications

References 9 publications

Temporal-Logic-Based Intermittent, Optimal, and Safe Continuous-Time Learning for Trajectory Tracking

Temporal-Logic-Based Intermittent, Optimal, and Safe Continuous-Time Learning for Trajectory Tracking

Dynamic intermittent Q‐learning–based model‐free suboptimal co‐design of ‐stabilization

Sparse event‐triggered control of linear systems

Contact Info

Product

Resources

About