Adaptive reinforcement learning in control design for cooperating manipulator systems

Abstract

In this paper, an optimal motion/force hybrid control strategy based on adaptive reinforcement learning (ARL) is proposed for cooperating manipulator systems.The optimal trajectory control and constraint force factor control, by using the Moore-Penrose pseudoinverse, are addressed to design the controller corresponding to the manipulator dynamic model. In addition, a frame of a different auxiliary term and an appropriate state-variable vector are presented to address the non-autonomous closed system with a time-varying desired trajectory. The simultaneous actor/critic algorithm is implemented by optimizing the squared Bellman difference to be computed from the error of control policy and optimal control input. Moreover, the constraint force factor controller is discussed by a nonlinear technique after achieving the result of the constraint force factor. The tracking and convergence of ARL-based optimal motion/force hybrid control strategy are validated in closed-loop systems by a proposed Lyapunov function candidate. Finally, simulation results are implemented on a constrained system using three manipulators to verify the physical implementation of the presented optimal motion/force hybrid-tracking control strategy.

Manipulator trajectory tracking based on adaptive fuzzy sliding mode control

“…The proposed algorithm was evaluated in the literature using different intelligent techniques, including genetic algorithms and cuckoo search algorithms with PID, proving its superiority in following trajectories. Dao and Liu 53 proposed an adaptive reinforcement learning‐based optimal motion/force hybrid strategy for a collaborative robotic. The Moore‐Penrose pseudo‐inverse algorithm is used for optimal trajectory and binding factor, and a controller corresponding to the robotic arm dynamics model is designed.…”

Deep Reinforcement Learning for Integrated Non-Linear Control of Autonomous UAVs

“…In current research, RL based dynamic Programming has been used for evaluating the Optimal Reward Function to be incorporated with conventional DDPG Algorithm, which makes it novel and unique as compared to other approaches. In another paper, Dao et al [26] employed Actor Critic Network through ARL. This work is in accordance with the current work in which both DDPG and PPO have been used which are also from a family of Actor-Critic networks, with prime difference of a Reward Function.…”

Formation control scheme with reinforcement learning strategy for a group of multiple surface vehicles

“…Therefore, many scholars investigate control designs for each agent with the Euler-Lagrange (EL) dynamic equation, as well as MAS control systems. However, due to the difficulty in unifying the optimal control task and the trajectory tracking problem, there exist a few optimal control approaches, [1][2][3][4] but many typical robust adaptive control schemes [5][6][7][8][9][10] for each agent are represented under the EL dynamic equation. Several authors utilize the description of the Hamiltonian function to find the updating laws of actor/critic (AC) neural networks (NN) [1][2][3][4] after establishing tracking error models under a time-invariant representation for cooperating manipulators, 1,2 surface vehicles (SVs), 3 or wheeled mobile robots (WMRs).…”

“…However, due to the difficulty in unifying the optimal control task and the trajectory tracking problem, there exist a few optimal control approaches, [1][2][3][4] but many typical robust adaptive control schemes [5][6][7][8][9][10] for each agent are represented under the EL dynamic equation. Several authors utilize the description of the Hamiltonian function to find the updating laws of actor/critic (AC) neural networks (NN) [1][2][3][4] after establishing tracking error models under a time-invariant representation for cooperating manipulators, 1,2 surface vehicles (SVs), 3 or wheeled mobile robots (WMRs). 4 On the other hand, based on the foundation of traditional nonlinear control laws, some remarkable progress is mentioned, including the extension of the event-triggered strategy 6 and the development of special transformation functions in adaptive backstepping methods to handle constraint requirements.…”