Optimizing Reinforcement Learning Control Model in Furuta Pendulum and Transferring it to Real-World

Abstract

Reinforcement learning does not require explicit robot modeling as it learns on its own based on data, but it is temporal and spatial constraints when transfer it to real-world environments. In this research, we train a balancing Furuta pendulum problem, which is difficult to modeling, in a virtual environment (Unity) and transfer to real-world. The balancing Furuta pendulum problem is maintaining balance of the pendulum's end effector at a vertical position. We resolved the temporal and spatial by performing reinforcement learning in a virtual environment. Furthermore, we design a novel reward function that enabled faster and more stable problem-solving compared to existing two reward functions. We validate each reward function by applying it to soft actor critic (SAC) and proximal policy optimization (PPO). The experimental result shows that cosine reward function is trained faster and more stable. Finally, SAC algorithm model using a cosine reward function in the virtual environment is an optimized controller. Additionally, we evaluated the robustness of this model by transferring it to the real environment.