Deep Reinforcement Learning-Based Control of Stewart Platform With Parametric Simulation in ROS and Gazebo

Abstract

The Stewart platform is an entirely parallel robot with mechanical differences from typical serial robotic manipulators, which has a wide application area ranging from flight and driving simulators to structural test platforms. This work concentrates on learning to control a complex model of the Stewart platform using state-of-the-art deep reinforcement algorithms (DRL). In this regard, to enhance the reliability of the learning performance and to have a test bed capable of mimicking the behavior of the system completely, a precisely designed simulation environment is presented. Therefore, we first design a parametric representation for the kinematics of the Stewart platform in Gazebo and robot operating system (ROS) and integrate it with a Python class to conveniently generate the structures in simulation description format (SDF). Then, to control the system, we benefit from three DRL algorithms: the asynchronous advantage actor-critic (A3C), the Deep Deterministic Policy Gradient(DDPG), and the Proximal Policy Optimization (PPO) to learn the control gains of a Proportional Integral Derivative (PID) controller for a given reaching task. We chose to apply these algorithms due to the Stewart platform's continuous action and state spaces, making them well-suited for our problem, where an exact controller tuning is a crucial task. The simulation results show that the DRL algorithms can successfully learn the controller gains, resulting in satisfactory control performance.

A high accuracy Stewart–lift platform based on a programmable logic controller – comparative case studies

“…The study in [13] appears to be similar to the previously presented idea. In these assignment authors present the control of a nonlinear Stewart platform and put more emphasis on artiőcial intelligence algorithms.…”

Addressing Challenges in Dynamic Modeling of Stewart Platform using Reinforcement Learning-Based Control Approach

“…1. Drawing of the kinematics and coordinate system of the Stewart platform [17] There are two coordinate systems, base B xyz and moving platform M xyz . Since we have six legs in the Stewart platform design, we have six attachment points in both the base and motion platforms.…”

“…We form RL and control blocks similar to the RL setup experimented in [17]. However, we change the action space completely from changing PID gains to applying force to each leg.…”

“…Within the literature, numerous neural network methodologies have been proposed to address the forward kinematics of the Stewart platform which is complex due to a set of high nonlinear equations [13]- [16]; but in terms of RL, there are few studies only. In a most recent study done by [17], deep RL algorithms to tune the gain parameters of a PID controller are used. This approach facilitated continuous learning and tuning Journal of Robotics and Control (JRC) ISSN: 2715-5072 118 of the controller's parameters.…”

“…In a second attempt to improve the work carried out in [17], we propose a hybrid RL algorithm to learn a dynamical model of the system, resulting in more sample efficiency, wellsuited to real-world applications like robotics [35], [36]. Even though model-free RL algorithms have succeeded in many areas, like video games and robotics, high sample complexity can limit the usage of model-free algorithms to simulated environments [31], [37]- [39].…”