Reinforcement learning (RL) offers powerful algorithms to search for optimal controllers of systems with nonlinear, possibly stochastic dynamics that are unknown or highly uncertain. This review mainly covers artificial-intelligence approaches to RL, from the viewpoint of the control engineer. We explain how approximate representations of the solution make RL feasible for problems with continuous states and control actions. Stability is a central concern in control, and we argue that while the control-theoretic RL subfield called adaptive dynamic programming is dedicated to it, stability of RL largely remains an open question. We also cover in detail the case where deep neural networks are used for approximation, leading to the field of deep RL, which has shown great success in recent years. With the control practitioner in mind, we outline opportunities and pitfalls of deep RL; and we close the survey with an outlook that -among other things -points out some avenues for bridging the gap between control and artificial-intelligence RL techniques.
This paper investigates stability of nonlinear con trol systems under intermittent information. Building on the small-gain theorem, we develop self-triggered control yielding stable closed-loop systems. We take the violation of the small gain condition to be the triggering event, and develop a sam pling policy that precludes this event by executing the control law with up-to-date information. Based on the properties of the external inputs to the plant, the developed sampling policy yields regular stability, asymptotic stability and Lp-stability.Control loops are modeled as interconnections of hybrid sys tems, and novel results on Lp-stability of hybrid systems are presented. Prediction of the triggering event is achieved by employing Lp-gains over a finite horizon. In addition, Lp-gains over a finite horizon produce larger intersampling intervals when compared with standard Lp-gains. Furthermore, a novel approach for calculation of Lp-gains over a finite horizon is devised. Finally, our approach is successfully applied to a trajectory tracking control system.
In this paper, we study event-triggered data scheduling for stochastic multi-loop control systems communicating over a shared network with communication uncertainties. We introduce a novel dynamic scheduling scheme which allocates the channel access according to an error-dependent policy. The proposed scheduler deterministically excludes subsystems with lower error values from the medium access competition in favor of those with larger errors. Subsequently, the scheduler probabilistically allocates the communication resource to the eligible entities. We model the overall networkinduced error as a homogeneous Markov chain and show its boundedness in expectation over a multi time-step horizon. In addition, analytical upper bound for the associated average cost is derived. Furthermore, we show that our proposed policy is robust against packet dropouts. Numerical results demonstrate a significant performance improvement in terms of error level in comparison with periodic and random scheduling policies. 53rd IEEE Conference on Decision and Control
In this article, we consider a nonlinear process with delayed dynamics to be controlled over a communication network in the presence of disturbances and study robustness of the resulting closed-loop system with respect to network-induced phenomena such as sampled, distorted, delayed and lossy data as well as scheduling protocols. For given plant-controller dynamics and communication network properties (e.g., propagation delays and scheduling protocols), we quantify the control performance level (in terms of L p -gains) as the transmission interval varies. Maximally Allowable Transfer Interval (MATI) labels the greatest transmission interval for which a prescribed L p -gain is attained. The proposed methodology combines impulsive delayed system modeling with Lyapunov-Razumikhin techniques to allow for MATIs that are smaller than the communication delays. Other salient features of our methodology are the consideration of variable delays, corrupted data and employment of modelbased estimators to prolong MATIs. The present stability results are provided for the class of Uniformly Globally Exponentially Stable (UGES) scheduling protocols. The well-known Round Robin (RR) and Try-Once-Discard (TOD) protocols are examples of UGES protocols. Finally, two numerical examples are provided to demonstrate the benefits of the proposed approach. * D. Tolić is with Faculty of Electrical Engineering and Computing, University of Zagreb, Unska 3, 10000 Zagreb, Croatia. S. Hirche is with the Chair
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.