Crowd-Robot Interaction: Crowd-Aware Robot Navigation With Attention-Based Deep Reinforcement Learning

Abstract

This paper proposes an end-to-end deep reinforcement learning approach for mobile robot navigation with dynamic obstacles avoidance. Using experience collected in a simulation environment, a convolutional neural network (CNN) is trained to predict proper steering actions of a robot from its egocentric local occupancy maps, which accommodate various sensors and fusion algorithms. The trained neural network is then transferred and executed on a real-world mobile robot to guide its local path planning. The new approach is evaluated both qualitatively and quantitatively in simulation and realworld robot experiments. The results show that the map-based end-to-end navigation model is easy to be deployed to a robotic platform, robust to sensor noise and outperforms other existing DRL-based models in many indicators.

Effects of a Social Force Model Reward in Robot Navigation Based on Deep Reinforcement Learning

“…There are other works that use Imitation Learning approaches to learn policies [16,18,23]. In the Crowd-Robot Interaction (CRI) approach [3], in which is mainly based the navigation in this paper, Imitation Learning is combined with the Deep Reinforcement Learning task to obtain a policy (SARL) for robot navigation. For Imitation Learning, the policy used to control all the agents is ORCA.…”

“…The states for all the agents and obstacles are always known and used to calculate the value function using a neural network and the Temporal Difference algorithm of the CRI work [3]. With this value function the greedy policy is calculated, which gives in this case the robot's velocity(the optimal action) each timestep.…”

“…In this work two types of rewards are used to guide the robot to the correct behaviour. The first reward is the reward used in the CRI approach [3]:…”

Robot Navigation in Crowds by Graph Convolutional Networks With Attention Learned From Human Gaze

“…However, the model parameters need to be tuned for different application scenarios. More recent research has used deep reinforcement learning (RL) successfully to learn efficient policies that model the cooperation and interactions implicitly [9]- [11]. We propose to represent the interactions among humans and robot as a graph with both nodes and edges varying over time.…”

“…Previously proposed approaches differ primarily in the structure of the networks used to encode the robot-crowd state and to estimate the corresponding value. There are several limitations of current models, which cause their performance to degrade when the crowd density increases [9], [10]. First, existing models have considered only pairwise interactions between the robot and each human in the crowd.…”