Deep Reinforcement Learning for Event-Triggered Control

Abstract

Event-triggered control (ETC) methods can achieve high-performance control with a significantly lower number of samples compared to usual, time-triggered methods. These frameworks are often based on a mathematical model of the system and specific designs of controller and event trigger. In this paper, we show how deep reinforcement learning (DRL) algorithms can be leveraged to simultaneously learn control and communication behavior from scratch, and present a DRL approach that is particularly suitable for ETC. To our knowledge, this is the first work to apply DRL to ETC. We validate the approach on multiple control tasks and compare it to model-based event-triggering frameworks. In particular, we demonstrate that it can, other than many model-based ETC designs, be straightforwardly applied to nonlinear systems. Accepted final version.

Application of deep reinforcement learning to networked control systems with uncertain network delays

“…Moreover, we assume that network delays fluctuate randomly due to network routing, where their maximum values are known beforehand. Recently, DRL-based networked control methods have been proposed in [13,17]. However, in these researches, it is not assumed that there are network delays.…”

“…It is shown that the DDPG algorithm can solve more than 20 simulated physics tasks. As its applications to control problems, for example, Baumann et al proposed a method to design an event-triggered controller using the DDPG algorithm in [17] and Duan et al proposed the method to master the power grid voltage control problem using not only the DQN algorithm but also the DDPG algorithm in [18].…”

Event-triggered Pulse Control with Model Learning (if Necessary)

“…Event-triggered controllers can also be learned from data without learning a model. Such approaches are proposed, for example, in [12]- [15]. In contrast to those approaches, we use a specific control design and use learning to obtain accurate dynamic models.…”

Deep reinforcement learning for scheduling in large-scale networked control systems

“…Deep RL has been applied successfully to various control applications. Baumann et al (2018) applied the recent success of deep actor-critic algorithms in an event-triggered control scenario. In Lenz et al (2015), the authors combined system identification based on deep learning with model predictive control.…”