A mobile robot controller using reinforcement learning under scLTL specifications with uncertainties

Abstract

The control of a mobile robot is a well-studied problem in robotics that can be solved easily when performing simple tasks without uncertainties. However, uncertainties always exist when a mobile robot performs a series of tasks in the real-world. In this paper, we proposed a reinforcement learning (RL)-based controller for a mobile robot to perform a task with uncertainties.We consider the case where a task consists of several subtasks described by syntactically co-safe linear temporal logic (scLTL) specifications and each scLTL specification is transformed into a finite state automaton (FSA) that accepts all behavior satisfying the scLTL specification. A reinforcement learning with an FSA-encoder (RLwF) method is proposed to learn an optimal control policy in performing tasks in an environment with uncertainties. We propose an RL-based control method with uncertainties to learn rapidly an optimal policy against the uncertainties such as jammers that prevent a mobile robot from performing specified tasks. By simulation, we demonstrate that the proposed controller can learn an optimal policy and generate an optimal path to perform the designed tasks.

Fully cooperative games with state and input constraints using reinforcement learning based on control barrier functions

“…This paper focuses on investigating a safe RL that incorporates constraints on both the states and control inputs. The objective is for each player to find out a safe optimal control policy that stabilizes system (1), minimizes the value function (5), and satisfies the constraints (7) and (8). The problem can be formulated as follows:…”

“…In a related study [29], a method utilizing neural networks is proposed to obtain an initial stable control policy. However, this method relies on the traditional value function (5) and the reward function (27), which cannot guarantee the state and input constraints. In this study, we will present an effective method to obtain safe and stable control policies.…”

“…These days, reinforcement learning (RL) [1] has witnessed significant advancements in both research and practical applications, and it has been extensively applied in a wide range of fields such as autonomous navigation [2], aero-engine [3], and industrial robots [4,5]. For addressing the optimal control problems, RL is demonstrated to be effective and available in many works [6][7][8].…”

A system decomposition method for region tracking control of a non‐holonomic mobile robot with dynamic parameter uncertainties

“…On the other hand, the wheeled mobile robot is a classic mechanical system with the non-holonomic constraint, due to its constraints, uncertainties, and non-linearities. The mobile robot system poses challenges in the design of its motion control to achieve robust tracking control tasks, which is also the reason why many efforts have been made by researchers [3][4][5][6][7][8].…”