Simultaneous locomotion and manipulation control of quadruped robots using reinforcement learning-based adaptive fractional-order sliding-mode control

Abstract

This paper investigates a model-free reinforcement learning-based approach that enables the quadruped robot to manipulate objects while maintaining its balance and dynamic stability during walking. At first, the dynamics of quadruped robots in two sub-spaces, position control space and force control space, are developed. Then, a new long-term performance index is introduced, and a radial basis function neural network as a critic network is presented to estimate the unobtainable long-term performance index. Based on the exported reinforcement signal, the actor neural network is introduced to generate the feedforward compensation term to cope with the nonlinear dynamics and the system uncertainties. The robustness of the actor-critic reinforcement learning algorithm is enhanced by using a fractional-order sliding-mode controller in the closed-loop system. The online adaptive laws for both the critic and actor-network weights are obtained using the Lyapunov stability theory. As a result, the uniformly ultimately boundedness of the position and the force tracking errors are proven. Finally, numerical simulations are conducted to illustrate the feasibility and effectiveness of the proposed adaptive actor-critic learning-based control scheme.

Deep reinforcement learning-assisted extended state observer for run-to-run control in the semiconductor manufacturing process

“…Deep reinforcement learning (DRL), which integrates the perceptual capabilities of deep learning and the decision-making capabilities of RL, has been widely adopted in complex environments (Babu et al, 2023;Khodamipour et al, 2021). Building on the endto-end learning approach, DRL has been increasingly employed in the control theory (Arulkumaran et al, 2017;Farid, 2023;Li et al, 2023), such as parameter optimization of proportional-integral-differential (PID) controllers (Gheisarnejad and Khooban, 2020;Lawrence et al, 2022). The main idea is to use DRL as an adaptive strategy to achieve autotuning of PID parameters.…”