Visual Reference of Ambiguous Objects for Augmented Reality-Powered Human-Robot Communication in a Shared Workspace

Gao, Peng; Reily, Brian; Paul, Savannah; Zhang, Hao

doi:10.1007/978-3-030-49695-1_37

Cited by 8 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Collaborative perception is a fundamental capability in collaborative robotics for robots and other agents including humans in a collaborative team to share information of the surrounding environment thus achieving shared situational awareness among the teammates. Collaborative perception has been widely applied in a variety of real-world applications including human-robot collaborative assembly [18,20], multi-robot search and rescue [1,45], and connected autonomous driving [19,49]. Correspondence identification is defined as a problem to identify the same objects observed by multiple agents in their own fields of view, which is considered an essential component to enable collaborative perception [14,17,43].…”

Section: Introductionmentioning

confidence: 99%

Bayesian Deep Graph Matching for Correspondence Identification in Collaborative Perception

Gao¹,

Zhang²

2021

Robotics: Science and Systems XVII

Self Cite

View full text Add to dashboard Cite

Correspondence identification is essential for multirobot collaborative perception, which aims to identify the same objects in order to ensure consistent references of the objects by a group of robots/agents in their own fields of view. Although recent deep learning methods have shown encouraging performance on correspondence identification, they suffer from two shortcomings, including the inability to address non-covisibility in collaborative perception that is caused by occlusion and limited fields of view of the agents, and the inability to quantify and reduce uncertainty to improve correspondence identification. To address both issues, we propose a novel uncertainty-aware deep graph matching method for correspondence identification in collaborative perception. Our new approach formulates correspondence identification as a deep graph matching problem, which identifies correspondences based upon graph representations that are constructed from the agents' observations. We introduce a novel deep graph matching network under the Bayesian framework to explicitly quantify uncertainty in the identified correspondences. In addition, we design a novel loss function that explicitly reduces correspondence uncertainty and perceptual non-covisibility during learning. We evaluate our approach in the robotics applications of collaborative assembly and multi-robot coordination using high-fidelity simulations and physical robots. Experiments have shown that, through addressing both uncertainty and non-covisibility, our approach achieves the state-of-the-art performance of correspondence identification.

show abstract

Section: Introductionmentioning

confidence: 99%

Bayesian Deep Graph Matching for Correspondence Identification in Collaborative Perception

Gao¹,

Zhang²

2021

Robotics: Science and Systems XVII

Self Cite

View full text Add to dashboard Cite

show abstract

“…{gaopeng, hzhang}@mines.edu 2 Rui Guo and Hongsheng Lu are with Toyota Motor North America, Mountain View, CA 94043. {rui.guo, hongsheng.lu}@toyota.com locating street objects by connected vehicles [12], [13], multiview collaborative localization is also applied to various realworld applications, such as search and rescue [14], [15] and manufacture [16], [17].…”

Section: Introductionmentioning

confidence: 99%

Multi-view Sensor Fusion by Integrating Model-based Estimation and Graph Learning for Collaborative Object Localization

Gao¹,

Guo²,

Lu³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Collaborative object localization aims to collaboratively estimate locations of objects observed from multiple views or perspectives, which is a critical ability for multi-agent systems such as connected vehicles. To enable collaborative localization, several model-based state estimation and learning-based localization methods have been developed. Given their encouraging performance, model-based state estimation often lacks the ability to model the complex relationships among multiple objects, while learning-based methods are typically not able to fuse the observations from an arbitrary number of views and cannot well model uncertainty. In this paper, we introduce a novel spatiotemporal graph filter approach that integrates graph learning and model-based estimation to perform multi-view sensor fusion for collaborative object localization. Our approach models complex object relationships using a new spatiotemporal graph representation and fuses multi-view observations in a Bayesian fashion to improve location estimation under uncertainty. We evaluate our approach in the applications of connected autonomous driving and multiple pedestrian localization. Experimental results show that our approach outperforms previous techniques and achieves the state-of-the-art performance on collaboration localization.

show abstract

Designing a Shared Workspace for Learning Using Augmented Reality and Social Robots

Pasalidou

Fachantidis

2022

Communications in Computer and Information Science

View full text Add to dashboard Cite

Visual Reference of Ambiguous Objects for Augmented Reality-Powered Human-Robot Communication in a Shared Workspace

Cited by 8 publications

References 14 publications

Bayesian Deep Graph Matching for Correspondence Identification in Collaborative Perception

Bayesian Deep Graph Matching for Correspondence Identification in Collaborative Perception

Multi-view Sensor Fusion by Integrating Model-based Estimation and Graph Learning for Collaborative Object Localization

Designing a Shared Workspace for Learning Using Augmented Reality and Social Robots

Contact Info

Product

Resources

About