Heuristic and deep reinforcement learning-based PID control of trajectory tracking in a ball-and-plate system

Abstract

The manual tuning of controller parameters, for example, tuning proportional integral derivative (PID) gains often relies on tedious human engineering. To curb the aforementioned problem, we propose an artificial intelligence-based deep reinforcement learning (RL) PID controller (three variants) compared with genetic algorithm-based PID (GA-PID) and classical PID; a total of five controllers were simulated for controlling and trajectory tracking of the ball dynamics in a linearized ball-and-plate (B&P) system. For the experiments, we trained novel variants of deep RL-PID built from a customized deep deterministic policy gradient (DDPG) agent (by modifying the neural network architecture), resulting in two new RL agents (DDPG-FC-350-R-PID & DDPG-FC-350-E-PID). Each of the agents interacts with the environment through a policy and a learning algorithm to produce a set of actions (optimal PID gains). Additionally, we evaluated the five controllers to assess which method provides the best performance metrics in the context of the minimum index in predictive errors, steady-state-error, peak overshoot, and timeresponses. The results show that our proposed architecture (DDPG-FC-350-E-PID) yielded the best performance and surpasses all other approaches on most of the evaluation metric indices. Furthermore, an appropriate training of an artificial intelligencebased controller can aid to obtain the best path tracking.

Photovoltaic System MPPT Evaluation Using Classical, Meta-Heuristics, and Reinforcement Learning-Based Controllers: A Comparative Study

“…The deep reinforcement learning architecture proposed is motivated by [37], which yielded the best performance than the heuristic or classical methods when tracking the ball dynamics in a ball and plate system. In the current study, we modify the architecture to deal with the MPP tracking task.…”

“…The layers containing {300, 2} network nodes employ exponential linear unit (ELU) activation function, which can be defined mathematically using Eq. 18 [37], [38].…”

Path-Following and Obstacle Avoidance Control of Nonholonomic Wheeled Mobile Robot Based on Deep Reinforcement Learning

“…It is a learning paradigm concerned with learning to control a system in order to maximize a cumulative expected reward performance measure that expresses a long-term objective [14] and can determine the optimal policy for decisions in a real environment. Recently, research into RL methods has been extended into multiple control fields such as trajectory tracking [15][16][17][18], path-following [19,20], etc. [21].…”

“…It is noted that RL is capable of coping with a control problem without knowing information about the objective dynamics and presents control with good performance under the influence of external disturbances [15,16,20]. Reinforcement learning can be combined with other classical control methods to solve tracking problems [17]. Considering the PID method, a Q-learning-PID control approach has been proposed to solve the trajectory tracking control problem for mobile robots, with better results than the single approach [18].…”

Tracking control of a ball on plate system using PID controller and Lead/Lag compensator with a double loop feedback scheme

“…The B&P system finds practical applications in several dynamic systems, such as robotics, rocket systems, and unmanned aerial vehicles. The enumerated systems are often expected to follow a time parameterized reference path [2]. Due to the BP complexity and it's nonlinearity, the mathematical model of the BPS presents uncertainties which increase the difficulty of designing a suitable controller.…”