Reinforcement Learning for Position Control Problem of a Mobile Robot

Abstract

Due to the increase in complexity in autonomous vehicles, most of the existing control systems are proving to be inadequate. Reinforcement Learning is gaining traction as it is posed to overcome these difficulties in a natural way. This approach allows an agent that interacts with the environment to get rewards for appropriate actions, learning to improve its performance continuously. The article describes the design and development of an algorithm to control the position of a wheeled mobile robot using Reinforcement Learning. One main advantage of this approach concerning traditional control algorithms is that the learning process is carried out automatically with a recursive procedure forward in time. Moreover, given the fidelity of the model for the particular implementation described in this work, the whole learning process can be carried out in simulation. This fact avoids damages to the actual robot during the learning stage. It shows that the position control of the robot (or similar specific tasks) can be done without the need to know the dynamic model of the system explicitly. Its main drawback is that the learning stage can take a long time to finish and that it depends on the complexity of the task and the availability of adequate hardware resources. This work provides a comparison between the proposed approach and traditional existing control laws in simulation and real environments. The article also discusses the main effects of using different controlled variables in the performance of the developed control law.

Criterios de desempeño para evaluar algoritmos de navegación de robots móviles: una revisión

“…Estos criterios de desempeño están basados en el error acumulativo y se pueden aplicar al seguimiento de trayectorias de referencia indicando el error durante todo el recorrido entre la trayectoria de referencia y la trayectoria real realizada por el robot. Estos índices también son usados para control de posición, distancia, orientación, formación de múltiples robots, etc (Caruntu et al, 2019), (Farias et al, 2020). Entre menor sea el error, mejor será la trayectoria recorrida, en consecuencia, mejor será el algoritmo de control.…”

“…Criterio: índice, métrica, medición, procedimiento Referencia % éxito en alcanzar la meta (Gridnev et al, 2017) (Graba et al, 2020) Integral del valor absoluto del error -integral absolute error (IAE) (Fernandes et al, 2017); (Caruntu et al, 2019); (Suarin et al, 2019); (Rivera et al, 2020); (Farias et al, 2020) Integral del valor absoluto del error ponderado en el tiempo -Integral time-weighted absolute error (ITAE) (Fernandes et al, 2017); (Suarin et al, 2019); (Farias et al, 2020) Integral del valor absoluto de la señal de control -Integral absolute signal control (IASC) (Fernandes et al, 2017) Integral del cuadrado del error -integral square error (ISE) (Perez et al, 2018); (Suarin et al, 2019); (Farias et al, 2020) integral del cuadrado del error por el tiempo -Integral time square error (ITSE) (Farias et al, 2020) Esfuerzo de control (Rivera et al, 2020) Seguridad (Marvel and Bostelman, 2014); (Munoz et al, 2014) Energía (Munoz et al, 2014); (Fernandes et al, 2017); (Stefek et al, 2020); (Graba et al, 2020); (Wei and Isler, 2020); (Hrubý et al, 2021)…”

Implementation of Q-Learning Algorithm on Arduino: Application to Autonomous Mobile Robot Navigation in COVID-19 Field Hospitals

“…In the Q-Learning paradigm, an agent interacts with the environment and executes a set of actions as illustrated by Fig. 4 [15]. However, the update of state-action values in Q-Learning is defined by the following equation [15]− [17] :…”

“…4 [15]. However, the update of state-action values in Q-Learning is defined by the following equation [15]− [17] :…”

Use of PID control during Education in Reinforcement Learning on Two Wheel Balance Robot

“…Farias and friends developed an algorithm to control the position of a wheeled mobile robot. The main advantage is learning procedure which is done automatically with recursive procedure [24]. Farias and friends proposed an 3D simulation environment for control the position of a wheeled mobile robot.…”