Digital Twin Simulations Based Reinforcement Learning for Navigation and Control of a Wheel-on-Leg Mobile Robot

Abstract

Hybrid mobile robots are able to function in a number of different modes of locomotion, which increases their capacity to overcome challenges and makes them appropriate for a wide range of applications. To be able to develop navigation techniques that make use of these improved capabilities, one must first have a solid grasp of the constraints imposed by each of those different modalities of locomotion. In this paper, we present a data-driven approach for evaluating the robots’ locomotion modes. To do this, we formalize the problem as a reinforcement learning task that is applied to a digital twin simulation of the mobile robot. The proposed method is demonstrated through the use of a case study that examines the capabilities of hybrid wheel-on-leg robot locomotion modes in terms of speed, slope ascent, and step obstacle climbing. First, a comprehensive explanation of the process of creating the digital twin of the mobile robot through the use of the Unity gaming engine is presented. Second, a description of the construction of three test environments is provided so that the aforementioned capabilities of the robot can be evaluated. In the end, Reinforcement Learning is used to evaluate the two types of locomotion that the mobile robot can utilize in each of these different environments. Corresponding simulations are conducted in the virtual environment and the results are analyzed.