Adaptive Reinforcement Learning-Enhanced Motion/Force Control Strategy for Multirobot Systems

Abstract

This paper presents an adaptive reinforcement learning- (ARL-) based motion/force tracking control scheme consisting of the optimal motion dynamic control law and force control scheme for multimanipulator systems. Specifically, a new additional term and appropriate state vector are employed in designing the ARL technique for time-varying dynamical systems with online actor/critic algorithm to be established by minimizing the squared Bellman error. Additionally, the force control law is designed after obtaining the computation of constraint force coefficient by the Moore–Penrose pseudo-inverse matrix. The tracking effectiveness of the ARL-based optimal control is verified in the closed-loop system by theoretical analysis. Finally, simulation studies are conducted on a system of three manipulators to validate the physical realization of the proposed optimal tracking control design.

A Direct Reinforcement Learning Approach for Nonautonomous Thermoacoustic Generator

“…Remark 1. It is worth emphasizing that unlike the work in [2,22] studying RL technique for time varying closed control system of robotics by indirect methods, the proposed method in this article develops the direct RL method for time varying TAG systems.…”

“…Assume that all parameters are linear and frequencyindependent. Ignoring hysteresis losses, the alternator's impedance in Figure 2 can be written as [22].…”

“…In this section, we consider the extension from classical RL algorithm [2,22,27] to time varying RL for the general class of time varying nonlinear systems as…”

“…According to dynamic programming (DP) principle [22,27,28], implementing the derivative with respect to time of optimal value J * (x(t), t) along the system trajectory (13), one can obtain the following equation:…”

“…(1) First, it is obviously different from [2,22] realizing RL procedure for robotic systems using differential equations; the structure of time varying RL is presented to obtain the optimal control for time varying TAG systems to be expressed by PDE. (2) Second, differential from researches of TAG systems [1], optimal control strategy is investigated with the related comparison to be discussed through theoretical analysis and simulation studies.…”