When2com: Multi-Agent Perception via Communication Graph Grouping

Liu, Yen‐Cheng; Tian, Junjiao; Glaser, Nathaniel; Kira, Zsolt

doi:10.1109/cvpr42600.2020.00416

Cited by 80 publications

(65 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This further shows the ability to scale the proposed approach to larger teams. It is possible to further reduce communication bandwidth by adopting smaller network encoder or learning communication groups as shown in [17]. For other sensing modalities, similar bandwidth trade-offs are expected.…”

Section: E Discussionmentioning

confidence: 98%

“…The degraded agent only connects with the selected agent to receive information. When2com [17] as its following work further investigates the multi-agent communication problem by constructing communication groups and deciding when to communicate based on the communication bandwidth. They validated their approach on semantic segmentation with noisy sensor inputs.…”

Section: Related Workmentioning

confidence: 99%

“…1) Photo-realistic Data: Airsim-MAP dataset was first proposed in When2com [17]. It contains images from five robots flying in a group in the city environment and rotating along their own z-axis with different angular velocities.…”

Section: Experiments a Datasetmentioning

confidence: 99%

“…In the noisy version of the dataset, Airsim-MAP applies Gaussian blurring with random kernel size from 1 to 100 and Gaussian noise [16]. Each camera has over 50% probability of being corrupted and each frame has at least one noisy image in five views [17]. We use this dataset to show that our approach is robust to sensor corruption.…”

Section: Experiments a Datasetmentioning

confidence: 99%

See 3 more Smart Citations

Multi-Robot Collaborative Perception With Graph Neural Networks

Zhou

Xiao²,

Zhou³

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Multi-robot systems such as swarms of aerial robots are naturally suited to offer additional flexibility, resilience, and robustness in several tasks compared to a single robot by enabling cooperation among the agents. To enhance the autonomous robot decision-making process and situational awareness, multi-robot systems have to coordinate their perception capabilities to collect, share, and fuse environment information among the agents efficiently to obtain context-appropriate information or gain resilience to sensor noise or failures. In this paper, we propose a general-purpose Graph Neural Network (GNN) with the main goal to increase, in multi-robot perception tasks, single robots' inference perception accuracy as well as resilience to sensor failures and disturbances. We show that the proposed framework can address multi-view visual perception problems such as monocular depth estimation and semantic segmentation. Several experiments both using photo-realistic and real data gathered from multiple aerial robots' viewpoints show the effectiveness of the proposed approach in challenging inference conditions including images corrupted by heavy noise and camera occlusions or failures.

show abstract

Section: E Discussionmentioning

confidence: 98%

Section: Related Workmentioning

confidence: 99%

Section: Experiments a Datasetmentioning

confidence: 99%

Section: Experiments a Datasetmentioning

confidence: 99%

See 2 more Smart Citations

Multi-Robot Collaborative Perception With Graph Neural Networks

Zhou

Xiao²,

Zhou³

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…Even though GNN-based architecture only requires robots communicating with certain hop neighbors, some of these communications may not be necessary. Some studies have presented parsimonious communication strategies that are selective in when to communicate with neighbors [62,63,109] and which neighbors to communicate with [110] to cutoff unnecessary communications among neighbors. Built on this idea, the third future avenue can be embedding parsimonious communication protocols into GNN architectures for multi-robot coordination.…”

Section: Parsimonious Communications For Multi-robot Coordination Using Gnnsmentioning

confidence: 99%

Multi-robot Coordination and Planning in Uncertain and Adversarial Environments

Zhou

Tokekar

2021

Curr Robot Rep

View full text Add to dashboard Cite

Purpose of Review Deploying a team of robots that can carefully coordinate their actions can make the entire system robust to individual failures. In this report, we review recent algorithmic development in making multi-robot systems robust to environmental uncertainties, failures, and adversarial attacks.Recent Findings We find the following three trends in the recent research in the area of multi-robot coordination: (1) resilient coordination to either withstand failures and/or attack or recover from failures/attacks; (2) risk-aware coordination to manage the trade-off risk and reward, where the risk stems due to environmental uncertainty; (3) Graph neural networks based coordination to learn decentralized multi-robot coordination policies. These algorithms have been applied to tasks such as formation control, task assignment and scheduling, search and planning, and informative data collection. SummaryIn order for multi-robot systems to become practical, we need coordination algorithms that can scale to large teams of robots dealing with dynamically changing, failure-prone, contested, and uncertain environments. There has been significant recent research on multi-robot coordination that has contributed resilient and risk-aware algorithms to deal with these issues and reduce the gap between theory and practice. Learning-based approaches have been seen to be promising, especially since they can learn who, when, and how to communicate for effective coordination. However, these algorithms have also been shown to be vulnerable to adversarial attacks, and as such developing learning-based coordination strategies that are resilient to such attacks and robust to uncertainties is an important open area of research.

show abstract

A Cordial Sync: Going Beyond Marginal Policies for Multi-agent Embodied Tasks

Jain

Weihs

Kolve

et al. 2020

Computer Vision – ECCV 2020

View full text Add to dashboard Cite

Autonomous agents must learn to collaborate. It is not scalable to develop a new centralized agent every time a task's difficulty outpaces a single agent's abilities. While multi-agent collaboration research has flourished in gridworld-like environments, relatively little work has considered visually rich domains. Addressing this, we introduce the novel task FurnMove in which agents work together to move a piece of furniture through a living room to a goal. Unlike existing tasks, FurnMove requires agents to coordinate at every timestep. We identify two challenges when training agents to complete FurnMove: existing decentralized action sampling procedures do not permit expressive joint action policies and, in tasks requiring close coordination, the number of failed actions dominates successful actions. To confront these challenges we introduce SYNC-policies (synchronize your actions coherently) and CORDIAL (coordination loss). Using SYNC-policies and CORDIAL, our agents achieve a 58% completion rate on FurnMove, an impressive absolute gain of 25 percentage points over competitive decentralized baselines. Our dataset, code, and pretrained models are available at https://unnat.github.io/cordial-sync.

show abstract

When2com: Multi-Agent Perception via Communication Graph Grouping

Cited by 80 publications

References 19 publications

Multi-Robot Collaborative Perception With Graph Neural Networks

Multi-Robot Collaborative Perception With Graph Neural Networks

Multi-robot Coordination and Planning in Uncertain and Adversarial Environments

A Cordial Sync: Going Beyond Marginal Policies for Multi-agent Embodied Tasks

Contact Info

Product

Resources

About