Dec-MCTS: Decentralized planning for multi-robot active perception

Best, Graeme; Cliff, Oliver M.; Patten, Timothy; Mettu, Ramgopal R.; Fitch, Robert

doi:10.1177/0278364918755924

Cited by 153 publications

(151 citation statements)

References 76 publications

(125 reference statements)

Supporting

Mentioning

150

Contrasting

Order By: Relevance

“…However, for multiagent scenarios, there is an additional challenge of the exponential growth of all the agents' action spaces for centralized methods [349]. One way to tackle this challenge within multiagent scenarios is the use of search parallelization [350,351]. Given more scalable planners, there is room for research in combining these techniques in MDRL settings.…”

Section: Open Questionsmentioning

confidence: 99%

A survey and critique of multiagent deep reinforcement learning

Hernández-Leal¹,

Kartal²,

Taylor³

2019

Auton Agent Multi-Agent Syst

421

240

View full text Add to dashboard Cite

Deep reinforcement learning (RL) has achieved outstanding results in recent years. This has led to a dramatic increase in the number of applications and methods. Recent works have explored learning beyond single-agent scenarios and have considered multiagent learning (MAL) scenarios. Initial results report successes in complex multiagent domains, although there are several challenges to be addressed. The primary goal of this article is to provide a clear overview of current multiagent deep reinforcement learning (MDRL) literature. Additionally, we complement the overview with a broader analysis: (i) we revisit previous key components, originally presented in MAL and RL, and highlight how they have been adapted to multiagent deep reinforcement learning settings. (ii) We provide general guidelines to new practitioners in the area: describing lessons learned from MDRL works, pointing to recent benchmarks, and outlining open avenues of research. (iii) We take a more critical tone raising practical challenges of MDRL (e.g., implementation and computational demands). We expect this article will help unify and motivate future research to take advantage of the abundant literature that exists (e.g., RL and MAL) in a joint effort to promote fruitful research in the multiagent community.$ Earlier versions of this work had the title: "Is multiagent deep reinforcement learning the answer or the question? A brief survey" arXiv:1810.05587v3 [cs.MA] 30 Aug 2019 Go [14,15], poker [16,17], and games of two competing teams, e.g., DOTA 2 [18] and StarCraft II [19].While different techniques and algorithms were used in the above scenarios, in general, they are all a combination of techniques from two main areas: reinforcement learning (RL) [20] and deep learning [21,22].RL is an area of machine learning where an agent learns by interacting (i.e., taking actions) within a dynamic environment. However, one of the main challenges to RL, and traditional machine learning in general, is the need for manually designing quality features on which to learn. Deep learning enables efficient representation learning, thus allowing the automatic discovery of features [21,22]. In recent years, deep learning has had successes in different areas such as computer vision and natural language processing [21,22]. One of the key aspects of deep learning is the use of neural networks (NNs) that can find compact representations in high-dimensional data [23].In deep reinforcement learning (DRL) [23,24] deep neural networks are trained to approximate the optimal policy and/or the value function. In this way the deep NN, serving as function approximator, enables powerful generalization. One of the key advantages of DRL is that it enables RL to scale to problems with high-dimensional state and action spaces. However, most existing successful DRL applications so far have been on visual domains (e.g., Atari games), and there is still a lot of work to be done for more realistic applications [25,26] with complex dynamics, which are not necessarily vision-based.DRL h...

show abstract

Section: Open Questionsmentioning

confidence: 99%

A survey and critique of multiagent deep reinforcement learning

Hernández-Leal¹,

Kartal²,

Taylor³

2019

Auton Agent Multi-Agent Syst

421

240

View full text Add to dashboard Cite

show abstract

“…By using Monte Carlo simulations to sample thousands of possible trajectories quickly, we can achieve good approximations of the values of possible actions. Decentralized Monte Carlo Tree Search (Dec-MCTS) leverages the power of MCTS to select an effective and compact sample space of action sequences for decentralized online planning [Best et al, 2019]. By sharing the decisions (intentions/plannings) with each other, i.e.…”

Section: Decentralized Monte Carlo Tree Searchmentioning

confidence: 99%

“…• Dec-MCTS: The agents run a searching tree individually and share the observing and deciding information with each other, which serves as the baseline in this paper [Best et al, 2019]. • Dec-MCTS-SP: Our approach that integrates sharing with prediction.…”

Section: Settingsmentioning

confidence: 99%

“…Communication is fundamental to coordinated behaviour, as robots need to develop decision strategies that take into account the actions of others [Best et al, 2018]. Through the cooperative planning process, multi-robot systems become effective for complex real-world tasks, including cooperative localisation, target tracking, object recognition, explo- ration, surveillance, and environmental monitoring [Best et al, 2019].…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we propose a novel decentralized planning approach by combining decision sharing and prediction in a more principled method. Our approach is based on the context of decentralized Monte Carlo Tree Search (Dec-MCTS) [Best et al, 2019] , which is called Dec-MCTS-SP. Its intuition is in analogy with human, when several players cooperate to play soccer, they need not talk frequently to let others know what they are going to do the next moment.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Integrating Decision Sharing with Prediction in Decentralized Planning for Multi-Agent Coordination under Uncertainty

Yang

Cai

et al. 2019

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

The performance of decentralized multi-agent systems tends to benefit from information sharing and its effective utilization. However, too much or unnecessary sharing may hinder the performance due to the delay, instability and additional overhead of communications. Aiming to a satisfiable coordination performance, one would prefer the cost of communications as less as possible. In this paper, we propose an approach for improving the sharing utilization by integrating information sharing with prediction in decentralized planning. We present a novel planning algorithm by combining decision sharing and prediction based on decentralized Monte Carlo Tree Search called Dec-MCTS-SP. Each agent grows a search tree guided by the rewards calculated by the joint actions, which can not only be sampled from the shared probability distributions over action sequences, but also be predicted by a sufficiently-accurate and computationally-cheap heuristics-based method. Besides, several policies including sparse and discounted UCT and DIY-bonus are leveraged for performance improvement. We have implemented Dec-MCTS-SP in the case study on multi-agent information gathering under threat and uncertainty, which is formulated as Decentralized Partially Observable Markov Decision Process (Dec-POMDP). The factored belief vectors are integrated into Dec-MCTS-SP to handle the uncertainty. Comparing with the random, auction-based algorithm and Dec-MCTS, the evaluation shows that Dec-MCTS-SP can reduce communication cost significantly while still achieving a surprisingly higher coordination performance.

show abstract

Ocean front detection and tracking using a team of heterogeneous marine vehicles

McCammon

Santos

Frantz

et al. 2021

Journal of Field Robotics

Self Cite

View full text Add to dashboard Cite

Ocean monitoring is an expensive and time consuming endeavor, but it can be made more efficient through the use of teams of autonomous robots. In this paper, we present a system for the autonomous identification and tracking of ocean fronts by coordinating the sampling efforts of a heterogeneous team of autonomous surface vehicles (ASVs) and autonomous underwater vehicles (AUVs). The primary contributions of this study are (1) our algorithm for performing autonomous coordination using general autonomy principles: Sequential Allocation Monte Carlo Tree Search (SA‐MCTS) which incorporates domain knowledge into the environmental estimation through both augmenting a standard Gaussian process with a nearest neighbors prior and planning in a drifting reference frame, (2) our decision support user interface to help human operators oversee the autonomous system, and (3) the demonstration of the system's operation in a 2‐week long deployment in the Gulf of Mexico using a heterogeneous team of four Slocum gliders and two robotic ocean surface samplers. With these contributions, we aim to bridge the gap between state of the art autonomy algorithms and marine vehicle planning methods that have been tested in large‐scale field trials. This paper presents the first deployment of a general, heuristic‐based, multi‐robot coordination algorithm for an extended sampling mission.

show abstract

Dec-MCTS: Decentralized planning for multi-robot active perception

Cited by 153 publications

References 76 publications

A survey and critique of multiagent deep reinforcement learning

A survey and critique of multiagent deep reinforcement learning

Integrating Decision Sharing with Prediction in Decentralized Planning for Multi-Agent Coordination under Uncertainty

Ocean front detection and tracking using a team of heterogeneous marine vehicles

Contact Info

Product

Resources

About