Cooperative Multi-Robot Navigation in Dynamic Environment with Deep Reinforcement Learning

Han, Ruihua; Chen, Shengduo; Hao, Qi

doi:10.1109/icra40945.2020.9197209

Cited by 36 publications

(18 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RL-NRVO is the policy using the original information of robots as input instead of RVO vectors. This information includes relative positions/velocities and robot radiuses, as introduced in [13], i.e., o sur = [p x , p y , v x , v y , R]. Similarly, RL-LSTM is the policy which replaces the BiGRUs with LSTM to tackle the input information of robots with varying numbers.…”

Section: B Results and Discussionmentioning

confidence: 99%

“…However, the dimension of the input data for a neural network is required to be fixed. Thus, for the environment model with the time-varying number of surrounding robots, some approaches assume that the number of obstacles is a constant and has an upper limit [13]. RNNs are able to tackle a variable number of moving obstacles, such as in GA3C-CADRL [11], where the exteroceptive measurements at each step are rearranged as the sequential input data through the long shortterm memory (LSTM) module to produce a fixed-size feature vector of the environment.…”

Section: B Deep Reinforcement Learningmentioning

confidence: 99%

“…There is still no systematic method to design rewards according to the observations for representing the collision risk precisely and guiding robots to achieve reciprocal collision avoidance (RCA) behaviors. Some methods use a set of vectors containing positions and velocities from multiple robots as the policy input [12], [13], which does not directly describe the collision avoidance interaction constraints among robots and hence demands extra capabilities of DRL networks to derive those constraints. To map the state of varying number of robots to actions, some approaches use recurrent neural networks (RNNs) to extract invariant features from the sequential input and output the value function to select optimal actions from a discrete action space [11], [14], but the unidirectional RNNs tend to focus on the recent input robot information instead of the information of all those robots.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Reinforcement Learned Distributed Multi-Robot Navigation with Reciprocal Velocity Obstacle Shaped Rewards

Ren¹,

Chen²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The challenges to solving the collision avoidance problem lie in adaptively choosing optimal robot velocities in complex scenarios full of interactive obstacles. In this paper, we propose a distributed approach for multi-robot navigation which combines the concept of reciprocal velocity obstacle (RVO) and the scheme of deep reinforcement learning (DRL) to solve the reciprocal collision avoidance problem under limited information. The novelty of this work is threefold: (1) using a set of sequential VO and RVO vectors to represent the interactive environmental states of static and dynamic obstacles, respectively; (2) developing a bidirectional recurrent module based neural network, which maps the states of a varying number of surrounding obstacles to the actions directly; (3) developing a RVO area and expected collision time based reward function to encourage reciprocal collision avoidance behaviors and trade off between collision risk and travel time. The proposed policy is trained through simulated scenarios and updated by the actor-critic based DRL algorithm. We validate the policy in complex environments with various numbers of differential drive robots and obstacles. The experiment results demonstrate that our approach outperforms the state-of-art methods and other learning based approaches in terms of the success rate, travel time, and average speed. Source code of this approach is available at https://github.com/ hanruihua/rl_rvo_nav.

show abstract

Section: B Results and Discussionmentioning

confidence: 99%

Section: B Deep Reinforcement Learningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Reinforcement Learned Distributed Multi-Robot Navigation with Reciprocal Velocity Obstacle Shaped Rewards

Ren¹,

Chen²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…RL is being extensively used for the navigation of ground robots by mapping raw sensor measurements to its navigation commands for obstacle avoidance (Fan et al, 2020). For instance, Han et al (2020) trained a homogenous multi-agent system of navigating ground robots that used proximal policy optimization to maximize target and obstacle avoidance rewards based on their poses and velocities. This approach is suitable for the navigation control of mobile robots to various machines on a manufacturing shop floor.…”

Section: Iced21mentioning

confidence: 99%

A Multi-Agent Reinforcement Learning Framework for Intelligent Manufacturing With Autonomous Mobile Robots

et al. 2021

View full text Add to dashboard Cite

Intelligent manufacturing (IM) embraces Industry 4.0 design principles to advance autonomy and increase manufacturing efficiency. However, many IM systems are created ad hoc, which limits the potential for generalizable design principles and operational guidelines. This work offers a standardizing framework for integrated job scheduling and navigation control in an autonomous mobile robot driven shop floor, an increasingly common IM paradigm. We specifically propose a multi-agent framework involving mobile robots, machines, humans. Like any cyberphysical system, the performance of IM systems is influenced by the construction of the underlying software platforms and the choice of the constituent algorithms. In this work, we demonstrate the use of reinforcement learning on a sub-system of the proposed framework and test its effectiveness in a dynamic scenario. The case study demonstrates collaboration amongst robots to maximize throughput and safety on the shop floor. Moreover, we observe nuanced behavior, including the ability to autonomously compensate for processing delays, and machine and robot failures in real time.

show abstract

“…Lin et al [10] aim to navigate the geometric center of a robot team to reach waypoints and develop an end-to-end policy shared among the robots that consider raw laser data inputs and position information of other robots. Han et al [22] also consider observable states of other robots and dynamic obstacles with Gaussian noises. They propose a greedy target allocation method to efficiently navigate the robots to multiple targets.…”

Section: Related Workmentioning

confidence: 99%

Decentralized Global Connectivity Maintenance for Multi-Robot Navigation: A Reinforcement Learning Approach

Li¹,

Jie²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

The problem of multi-robot navigation of connectivity maintenance is challenging in multi-robot applications. This work investigates how to navigate a multi-robot team in unknown environments while maintaining connectivity. We propose a reinforcement learning (RL) approach to develop a decentralized policy, which is shared among multiple robots. Given range sensor measurements and the positions of other robots, the policy aims to generate control commands for navigation and preserve the global connectivity of the robot team. We incorporate connectivity concerns into the RL framework as constraints and introduce behavior cloning to reduce the exploration complexity of policy optimization. The policy is optimized with all transition data collected by multiple robots in random simulated scenarios. We validate the effectiveness of the proposed approach by comparing different combinations of connectivity constraints and behavior cloning. We also show that our policy can generalize to unseen scenarios in both simulation and holonomic robots experiments.

show abstract

Cooperative Multi-Robot Navigation in Dynamic Environment with Deep Reinforcement Learning

Cited by 36 publications

References 15 publications

Reinforcement Learned Distributed Multi-Robot Navigation with Reciprocal Velocity Obstacle Shaped Rewards

Reinforcement Learned Distributed Multi-Robot Navigation with Reciprocal Velocity Obstacle Shaped Rewards

A Multi-Agent Reinforcement Learning Framework for Intelligent Manufacturing With Autonomous Mobile Robots

Decentralized Global Connectivity Maintenance for Multi-Robot Navigation: A Reinforcement Learning Approach

Contact Info

Product

Resources

About