A Novel Behavioral Strategy for RoboCode Platform Based on Deep Q‐Learning

Abstract

This paper addresses a new machine learning-based behavioral strategy using the deep Q-learning algorithm for the RoboCode simulation platform. According to this strategy, a new model is proposed for the RoboCode platform, providing an environment for simulated robots that can be programmed to battle against other robots. Compared to Atari Games, RoboCode has a fairly wide set of actions and situations. Due to the challenges of training a CNN model for such a continuous action space problem, the inputs obtained from the simulation environment were generated dynamically, and the proposed model was trained by using these inputs. The trained model battled against the predefined rival robots of the environment (standard robots) by cumulatively benefiting from the experience of these robots. The comparison between the proposed model and standard robots of RoboCode Platform was statistically verified. Finally, the performance of the proposed model was compared with machine learning based-customized robots (community robots). Experimental results reveal that the proposed model is mostly superior to community robots. Therefore, the deep Q-learning-based model has proven to be successful in such a complex simulation environment. It should also be noted that this new model facilitates simulation performance in adaptive and partially cluttered environments.

Enhancing Stability and Performance in Mobile Robot Path Planning with PMR-Dueling DQN Algorithm

“…Recently, the use of deep reinforcement learning (DRL) algorithms as the fundamental control [ 26 ], navigation [ 27 ], localization [ 28 ], and planning [ 29 ] systems has gained popularity. The flexibility of artificial neural networks, along with the capacity to simulate, train, and utilize them as end-to-end solutions, has garnered interest from the research community.…”

A Low-Cost Q-Learning-Based Approach to Handle Continuous Space Problems for Decentralized Multi-Agent Robot Navigation in Cluttered Environments

“…[32], [36] A more recent decentralized navigation research is done in [37] utilizing Neural networks, however, the proposed method utilizes a centralized unit for local communication and sharing learned features by nearby agents so it is not fully decentralized. The main subject of this research "Q-Learning" is utilized in diverse kinds of problems, such as inventory management [38], behavior design or navigation in robotics [12], [39], [40] and game play [41]- [43], and it is typically utilized in continuous state space problems by using parametric function approximation methods [40], [44]. Authors in [51] address the problem of coordinating multiagents in a decentralized fashion.…”