Reactive navigation under a fuzzy rules-based scheme and reinforcement learning for mobile robots

Abstract

Robot navigation allows mobile robots to navigate among obstacles without hitting them and reaching the specified goal point. In addition to preventing collisions, it is also essential for mobile robots to sense and maintain an appropriate battery power level at all times to avoid failures and non-fulfillment with their scheduled tasks. Therefore, selecting the proper time to recharge the batteries is crucial to address the navigation algorithm design for the robot’s prolonged autonomous operation. In this paper, a machine learning algorithm is used to ensure the extended robot autonomy based on a reinforcement learning method combined with a fuzzy inference system. The proposal enables a mobile robot to learn whether to continue through its path toward the destination or modify its course on the fly, if necessary, to proceed toward the battery charging station, based on its current state. The proposal performs a flexible behavior to choose an action that allows a robot to move from a starting to a destination point, guaranteeing battery charge availability. This paper shows the obtained results using an approach with thirty-six states and its reduction with twenty states. The conducted simulations show that the robot requires fewer training epochs to achieve ten consecutive successes in the fifteen proposed scenarios than traditional reinforcement learning methods exhibit. Moreover, in four scenarios, the robot ends up with a battery level above 80%, that value is higher than the obtained results with two deterministic methods.

Dynamic Obstacle Avoidance and Path Planning through Reinforcement Learning

“…RL is especially well suited for use on mobile robots. This is because RL enables mobile robots to learn from their environment and adapt to changing conditions, which is necessary for autonomous navigation and decision-making; this is confirmed by [6]. RL algorithms can also enhance the operational efficiency of mobile robots by determining the best course of action in real-time [18].…”

“…By choosing actions that would maximize the projected future rewards, the robot learns to navigate its environment. The most effective action is the one that, out of all the potential actions, has the highest Q value [6]. Q-learning is therefore a value-based approach.…”

“…In mobile robotics, Q-learning can be applied in various ways depending on the task. For instance, in navigation tasks [6], the robot can learn to select the optimal path to reach a target location by evaluating the expected reward of each possible action (such as moving forward or turning left/right) in each state (that is, the robot's current location as well as the orientation).…”

“…Complete independence from human labeling is the key advantage of reinforcement learning [2]. Furthermore, reinforcement learning is a promising strategy for creating autonomous systems that can adapt to changing environments and learn from their own experiences because the estimation of the action-value function is self-driven by the trialand-error interaction with the environment and takes the robot states as the input of the model [6]. However, the training process can be time-consuming and requires careful tuning of hyperparameters to achieve optimal performance.…”