Robust control of three‐degree‐of‐freedom spherical actuator based on deep reinforcement learning

Abstract

Multi‐degree‐of‐freedom (multi‐DOF) spherical actuators have been developed for the fields of robotics and industrial machineries. We have proposed an outer rotor type three‐DOF spherical actuator that can realize a high torque density. Each coil input current is calculated using a torque generating equation based on the torque constant matrix. The permanent magnet type actuators have a problem with generating unexpected cogging torque due to various manufacturing errors. Manufacturing errors mainly mean differences between the ideal dimensions at the motor design stage and the actual dimensions in mass production. In this case, the actuator would exceed the limitations of classical proportional‐integral‐differential (PID) controllers. Therefore, we propose a current compensator using reinforcement learning by introducing a deep neural network that is expected to improve the robustness of spherical actuators. This current compensator was applied to uncertainty problems such as manufacturing fluctuations of cogging torque. We examined the reward, which is the main parameter of deep reinforcement learning, and reduced the control error compared to classical PID controller and simple neural network (NN) controller. © 2022 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

Adaptive Sliding‐Mode Control for Permanent Magnet Spherical Actuator Based on Trajectory Re‐Planning

“…The traditional linear predictor predicts the future state of the controlled object through the expression shown in (23) to compensate for the influence of delay, but considering that most permanent magnet spherical actuators are equipped with sensors that can directly measure the speed of the output shaft, there is no need to use the velocity estimation method in (22). In addition, the position information of some sensors is obtained by integrating the measured velocity information.…”

“…The electrifying strategy used in the closed‐loop control system of the PMSA designed in this paper is a torque calculation method [22]. First, it is necessary to establish the torque characteristic model $$$ \mathbf{M}\left(\theta \right) $$$ of a single stator coil and a single rotor permanent magnet.…”

A Review of Deep Reinforcement Learning Approaches for Smart Manufacturing in Industry 4.0 and 5.0 Framework

“…This algorithm is an extension of DQN for continuous action spaces, and it is the first off-policy algorithm for this type of space, showing positive performance in the control of complex systems. Fusayasu et al (2022) [146] present a novel application of DDPG in the control of multi-degree-of-freedom spherical actuators, characterised by their difficult control due to their strong non-linearities of torque. DDPG achieves a highly accurate and robust control, outperforming PID and neural network controllers.…”