Mixed-Granularity Human-Swarm Interaction

Patel, Jayam; Xu, Yicong; Pinciroli, Carlo

doi:10.1109/icra.2019.8793261

Cited by 18 publications

(24 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Through enabling a human-friendly visualization of how a robot is perceiving its environment, an improved human-in-the-loop model can be achieved ( Sidaoui et al, 2019 ; Gong et al, 2017 ). The use of AR technology for robotics has been elevated by the aid of several tools, mainly Vuforia Engine ( Patel et al, 2019 ; Makita et al, 2021 ; Comes et al, 2021 ), RosSharp ( Kästner and Lambrecht, 2019 ; Rosen et al, 2019 ; Qiu et al, 2021 ), ARCore ( Zhang et al, 2019 ; Chacko et al, 2020 ; Mallik and Kapila, 2020 ), and ARKit ( Feigl et al, 2020 ; McHenry et al, 2021 ). ARCore and ARKit are tools that have enhanced the AR experience for motion tracking, environmental understanding, light estimation, among other features.…”

Section: Discussionmentioning

confidence: 99%

Augmented Reality Meets Artificial Intelligence in Robotics: A Systematic Review

Bassyouni

Elhajj

2021

Front. Robot. AI

View full text Add to dashboard Cite

Recently, advancements in computational machinery have facilitated the integration of artificial intelligence (AI) to almost every field and industry. This fast-paced development in AI and sensing technologies have stirred an evolution in the realm of robotics. Concurrently, augmented reality (AR) applications are providing solutions to a myriad of robotics applications, such as demystifying robot motion intent and supporting intuitive control and feedback. In this paper, research papers combining the potentials of AI and AR in robotics over the last decade are presented and systematically reviewed. Four sources for data collection were utilized: Google Scholar, Scopus database, the International Conference on Robotics and Automation 2020 proceedings, and the references and citations of all identified papers. A total of 29 papers were analyzed from two perspectives: a theme-based perspective showcasing the relation between AR and AI, and an application-based analysis highlighting how the robotics application was affected. These two sections are further categorized based on the type of robotics platform and the type of robotics application, respectively. We analyze the work done and highlight some of the prevailing limitations hindering the field. Results also explain how AR and AI can be combined to solve the model-mismatch paradigm by creating a closed feedback loop between the user and the robot. This forms a solid base for increasing the efficiency of the robotic application and enhancing the user’s situational awareness, safety, and acceptance of AI robots. Our findings affirm the promising future for robust integration of AR and AI in numerous robotic applications.

show abstract

Section: Discussionmentioning

confidence: 99%

Augmented Reality Meets Artificial Intelligence in Robotics: A Systematic Review

Bassyouni

Elhajj

2021

Front. Robot. AI

View full text Add to dashboard Cite

show abstract

“…The pilots in the study reported that the monitoring of the swarm was complex and that more swarm autonomy and control options are required. Patel et al [10] use an interface with augmented reality to specify the overall goal for the swarm with a mixture of robot and environment-oriented modalities. We use a more informative visualization compared to [9] and there is flexible autonomy that allows the operator to adjust the autonomy of the swarm by modifying the messages that are exchanged with the UAVs [11].…”

Section: Introductionmentioning

confidence: 99%

“…We use a more informative visualization compared to [9] and there is flexible autonomy that allows the operator to adjust the autonomy of the swarm by modifying the messages that are exchanged with the UAVs [11]. The operator can choose to not intervene and grant full autonomy to the swarm, which makes our study different from [10]. There are also other studies that focus on the autonomy level of the swarm [12,13], different human-swarm interaction methods [10,14,15] and visualization techniques for the human-swarm systems [8,16] but this paper advances the state of the art in the following ways:…”

Section: Introductionmentioning

confidence: 99%

“…The operator can choose to not intervene and grant full autonomy to the swarm, which makes our study different from [10]. There are also other studies that focus on the autonomy level of the swarm [12,13], different human-swarm interaction methods [10,14,15] and visualization techniques for the human-swarm systems [8,16] but this paper advances the state of the art in the following ways:…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Designing a User-Centered Interaction Interface for Human–Swarm Teaming

Soorati

Clark

Ghofrani

et al. 2021

Drones

View full text Add to dashboard Cite

A key challenge in human–swarm interaction is to design a usable interface that allows the human operators to monitor and control a scalable swarm. In our study, we restrict the interactions to only one-to-one communications in local neighborhoods between UAV-UAV and operator-UAV. This type of proximal interactions will decrease the cognitive complexity of the human–swarm interaction to O(1). In this paper, a user study with 100 participants provides evidence that visualizing a swarm as a heat map is more effective in addressing usability and acceptance in human–swarm interaction. We designed an interactive interface based on the users’ preference and proposed a controlling mechanism that allows a human operator to control a large swarm of UAVs. We evaluated the proposed interaction interface with a complementary user study. Our testbed and results establish a benchmark to study human–swarm interaction where a scalable swarm can be managed by a single operator.

show abstract

“…Our experimental evaluation is based on a study that involved 18 operators, grouped in teams of 2, and 9 real mobile robots involved in an object transport scenario. This paper takes advantage of our previous work on multi-granularity [17] interface design, in which we showed which interaction modalities [18] and which graphical elements of a user interface [19] provide the best performance in a multi-operator setting.…”

Section: Introductionmentioning

confidence: 99%

Direct and Indirect Communication in Multi-Human Multi-Robot Interaction

Patel,

Ramaswamy,

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

How can multiple humans interact with multiple robots? The goal of our research is to create an effective interface that allows multiple operators to collaboratively control teams of robots in complex tasks. In this paper, we focus on a key aspect that affects our exploration of the design space of human-robot interfaces -inter-human communication. More specifically, we study the impact of direct and indirect communication on several metrics, such as awareness, workload, trust, and interface usability. In our experiments, the participants can engage directly through verbal communication, or indirectly by representing their actions and intentions through our interface. We report the results of a user study based on a collective transport task involving 18 human subjects and 9 robots. Our study suggests that combining both direct and indirect communication is the best approach for effective multi-human / multi-robot interaction.

show abstract

Mixed-Granularity Human-Swarm Interaction

Cited by 18 publications

References 18 publications

Augmented Reality Meets Artificial Intelligence in Robotics: A Systematic Review

Augmented Reality Meets Artificial Intelligence in Robotics: A Systematic Review

Designing a User-Centered Interaction Interface for Human–Swarm Teaming

Direct and Indirect Communication in Multi-Human Multi-Robot Interaction

Contact Info

Product

Resources

About