On Distributed Implementation of Switch-Based Adaptive Dynamic Programming

Abstract

Switch-based adaptive dynamic programming (ADP) is an optimal control problem in which a cost must be minimized by switching among a family of dynamical modes. When the system dimension increases, the solution to switch-based ADP is made prohibitive by the exponentially increasing structure of the value function approximator and by the exponentially increasing modes. This technical correspondence proposes a distributed computational method for solving switch-based ADP. The method relies on partitioning the system into agents, each one dealing with a lower dimensional state and a few local modes. Each agent aims to minimize a local version of the global cost while avoiding that its local switching strategy has conflicts with the switching strategies of the neighboring agents. A heuristic algorithm based on the consensus dynamics and Nash equilibrium is proposed to avoid such conflicts. The effectiveness of the proposed method is verified via traffic and building test cases.

Reinforcement learning‐based robust optimal output regulation for constrained nonlinear systems with static and dynamic uncertainties

“…For simplicity, ( 14)-( 15) is rewritten as follows θ = S(𝜗, e, 𝜛), (18) ̇e = F(e) + G ū + D(𝜗, e, 𝜛), (19) where F(e) = ΨNe, G = b, D(𝜗, e, 𝜛) = g(z, e, 𝜛) + bΨ𝜂 is the uncertain part. | ū| ≤ ūM is the feedback controller, and ūM is its constraint band.…”

“…With the success of AlphaGo, RL has received considerable attention 16‐19 . Due to its universal approximation and self‐learning, RL has become a popular tool in dealing with optimal control problem 20‐22 .…”

“…11,12 To solve this problem, reinforcement learning (RL) was introduced, see References 13-15 and the references therein.With the success of AlphaGo, RL has received considerable attention. [16][17][18][19] Due to its universal approximation and self-learning, RL has become a popular tool in dealing with optimal control problem. [20][21][22] Up to now, many highlighted results have been published to develop the OORP.…”

Traffic Signal Control With Adaptive Online-Learning Scheme Using Multiple-Model Neural Networks

“…There are also many artificial intelligence approaches to addressing the traffic signal control problem, such as swarm intelligence [11,12], fuzzy logic algorithm [13,14,15], and reinforcement learning approaches [16,17,18]. For example, in [19,20] the authors use several adaptive dynamic programming approaches to develop an optimal traffic signal control scheme and identify the unknown traffic dynamics. In [21,22,23], some data-driven machine-learning approaches 2 are proposed for smart traffic signal control and optimal traffic management.…”

H ∞ Consensus Design and Online Scheduling for Multiagent Systems with Switching Topologies via Deep Reinforcement Learning

“…Deep learning can promote the considerable performance by effective nonlinear approximate ability, and reinforcement learning can explore the optimal control policy efficiently, which is widely applied in many areas. In the work of [43], the switch-based adaptive dynamic programming is proposed in optimal control problem to minimize the cost, which proposes a distributed computational method for adaptive dynamic programming. In [44], a noninteger PID controller is proposed based on deep reinforcement learning.…”