Representing Multi-Robot Structure through Multimodal Graph Embedding for the Selection of Robot Teams

Reily, Brian; Reardon, Christopher; Zhang, Hao

doi:10.1109/icra40945.2020.9197389

Cited by 14 publications

(5 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this work, we focus on the problem of embedding structural features of various robots through message passing neural networks [4]. Our goal of finding the robot design embedding differs from other existing tasks [8], which do not consider any kinematic constraints. This section will review the prior work that leverages GNN for robotics and then go over more recent works on multi-task learning.…”

Section: Related Workmentioning

confidence: 99%

Learning Robot Structure and Motion Embeddings using Graph Neural Networks

Kim¹,

Park²,

Choi³

et al. 2021

Preprint

View full text Add to dashboard Cite

We propose a learning framework to find the representation of a robot's kinematic structure and motion embedding spaces using graph neural networks (GNN). Finding a compact and low-dimensional embedding space for complex phenomena is a key for understanding its behaviors, which may lead to a better learning performance, as we observed in other domains of images or languages. However, although numerous robotics applications deal with various types of data, the embedding of the generated data has been relatively less studied by roboticists. To this end, our work aims to learn embeddings for two types of robotic data: the robot's design structure, such as links, joints, and their relationships, and the motion data, such as kinematic joint positions. Our method exploits the tree structure of the robot to train appropriate embeddings to the given robot data. To avoid overfitting, we formulate multi-task learning to find a general representation of the embedding spaces. We evaluate the proposed learning method on a robot with a simple linear structure and visualize the learned embeddings using t-SNE. We also study a few design choices of the learning framework, such as network architectures and message passing schemes.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning Robot Structure and Motion Embeddings using Graph Neural Networks

Kim¹,

Park²,

Choi³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, many others work to coordinate multiple robots to maximize sensor coverage, such as with power-and capability-limited robots used in swarms [12], or to identify corresponding objects between observations [13]. Similar approaches work by dividing swarms into subgroups to maximize the monitoring of multiple areas of interest [14], deploying [15] or dividing [16] heterogeneous teams of robots based on their sensing capabilities, or attempting to merge sensor observations to provide 'collective perception' [1]. In contrast to these approaches, we formulate our method to integrate object recognition as well as identifying the individual robots with the most discriminative observations.…”

Section: A Multi-robot Perceptionmentioning

confidence: 99%

Simultaneous View and Feature Selection for Collaborative Multi-Robot Perception

Reily,

Zhang

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Collaborative multi-robot perception provides multiple views of an environment, offering varying perspectives to collaboratively understand the environment even when individual robots have poor points of view or when occlusions are caused by obstacles. These multiple observations must be intelligently fused for accurate recognition, and relevant observations need to be selected in order to allow unnecessary robots to continue on to observe other targets. This research problem has not been well studied in the literature yet. In this paper, we propose a novel approach to collaborative multi-robot perception that simultaneously integrates view selection, feature selection, and object recognition into a unified regularized optimization formulation, which uses sparsity-inducing norms to identify the robots with the most representative views and the modalities with the most discriminative features. As our optimization formulation is hard to solve due to the introduced non-smooth norms, we implement a new iterative optimization algorithm, which is guaranteed to converge to the optimal solution. We evaluate our approach on multi-view benchmark datasets, a case-study in simulation, and on a physical multi-robot system. Experimental results demonstrate that our approach enables accurate object recognition and effective view selection as defined by mutual information.

show abstract

“…Accordingly, most homogeneous sensor coverage approaches address the problem of evenly distributing multiple robots spatially in an environment, as there is no need to consider individual capabilities [10], [11]. This has been accomplished through Voronoi distributions [12], decomposition of an environment into cells [13], estimating density functions [14], or representing an environment as a graph and utilizing graph partitioning methods to assign robots to regions [15]- [17] or teams [18]. Additionally, partitioning an environment has been done by calculating the information gain estimated from different regions [19], using a market-based system to assign robots based on information gain [20], or by planning paths that use greedy algorithms to maximize spatial coverage [21].…”

Section: Related Workmentioning

confidence: 99%

Adaptation to Team Composition Changes for Heterogeneous Multi-Robot Sensor Coverage

Reily¹,

Mott²,

Zhang³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

We consider the problem of multi-robot sensor coverage, which deals with deploying a multi-robot team in an environment and optimizing the sensing quality of the overall environment. As real-world environments involve a variety of sensory information, and individual robots are limited in their available number of sensors, successful multi-robot sensor coverage requires the deployment of robots in such a way that each individual team member's sensing quality is maximized. Additionally, because individual robots have varying complements of sensors and both robots and sensors can fail, robots must be able to adapt and adjust how they value each sensing capability in order to obtain the most complete view of the environment, even through changes in team composition. We introduce a novel formulation for sensor coverage by multi-robot teams with heterogeneous sensing capabilities that maximizes each robot's sensing quality, balancing the varying sensing capabilities of individual robots based on the overall team composition. We propose a solution based on regularized optimization that uses sparsity-inducing terms to ensure a robot team focuses on all possible event types, and which we show is proven to converge to the optimal solution. Through extensive simulation, we show that our approach is able to effectively deploy a multi-robot team to maximize the sensing quality of an environment, responding to failures in the multi-robot team more robustly than non-adaptive approaches.

show abstract

Representing Multi-Robot Structure through Multimodal Graph Embedding for the Selection of Robot Teams

Cited by 14 publications

References 35 publications

Learning Robot Structure and Motion Embeddings using Graph Neural Networks

Learning Robot Structure and Motion Embeddings using Graph Neural Networks

Simultaneous View and Feature Selection for Collaborative Multi-Robot Perception

Adaptation to Team Composition Changes for Heterogeneous Multi-Robot Sensor Coverage

Contact Info

Product

Resources

About