Situated communication for joint activity in human-robot teams

Abstract

H u m a n -a g e n t -R o b o t t e a m w o R k In real-life situations, robots often need to collaborate with humans. An experience and communication model supports the necessary shared activities.says something, we need to understand how it relates to the world around us: the when, what, and where. Our interpretation draws on a deep understanding of why someone says something and how we are supposed to act on it. What role does it play in the joint activity? How does it help us better understand each other and work together?The question here is what it takes to make a robot understand human communication and produce communication that humans can easily understand in a given context. This requires an understanding of human communication-yet robots see and experience the world differently from humans. Uncertainty and incompleteness pervade a robot's experience, but it must relate communication to that experience to figure out what the humans are discussing.In this article, we explain how we can model robot experience as a collection of representations that bridge the gap between low-level sensing and high-level conceptual structures and multiagent beliefs. Examples from urban search and rescue (USAR) scenarios illustrate the process. The examples are drawn from our experience in the NIFTi project, an ongoing European project to develop systems for human-robot teams to jointly explore urban disaster sites so as to make a situation assessment in the early phase of a disaster response. (NIFTi is the project acronym for the Natural Human-Robot Cooperation in Dynamic Environments project.) NIFTi teams are geographically distributed: the robots are deployed in the hot zone, while the humans can be either at a remote control post or in the field. NIFTi adopts a user-driven view, involving first responders in the entire development cycle. The project evaluates its systems each year in real-life conditions, at firstresponder training sites. Currently, NIFTi is at a stage in which several humans remotely collaborate with one or more semiautonomous unmanned ground vehicles (UGVs) using a multimodal GUI including spoken dialogue, and with an unmanned aerial vehicle (UAV) acting as a roving sensor. M any real-life applications require robots to carry out their tasks together with humans rather than acting autonomously. That requires communication to coordinate activities, work together, and act as a team. Team communication is about more than just words, however. When a team member 28 www.computer.org/intelligent iEEE iNTElliGENT SYSTEMS H u m a n -a g e n t -R o b o t t e a m w o R k Modeling ExperienceJoint activity between humans and robots takes place in a mutually experienced situation, in which all parties' experience provides the basis for how plans, actions, and their expected outcomes are individually understood. Communication plays a crucial role in building up a shared form of situational awareness to aid coordination and collaboration. Gary Klein and his colleagues outlined several basic requirements for turning a...

Conceptualisation of Human-on-the-Loop Haptic Teleoperation With Fully Autonomous Self-Driving Vehicles in the Urban Environment

“…HTSs and FA-SDVs need to collaborate and cooperate to achieve various tasks in this highly unpredicted urban roads. Joint activity between humans and robots takes place in a mutually experienced situation, in which all parties' experience provides the basis for how plans, actions, and their expected outcomes are individually understood [66]. Proper communication plays a crucial role in building up a shared form of situational awareness to aid coordination and collaboration [66].…”

“…Joint activity between humans and robots takes place in a mutually experienced situation, in which all parties' experience provides the basis for how plans, actions, and their expected outcomes are individually understood [66]. Proper communication plays a crucial role in building up a shared form of situational awareness to aid coordination and collaboration [66]. Timevarying delays are inevitable during teleoperation regarding the communication time delay problem, decision-making delays regarding QoE of HTSs and the evaluation of the voluminous sensor data and actuation time delays.…”

User Perceptions of Communicative and Task-competent Agents in a Virtual Basketball Game

“…For example, communication in sports often requires physical co-operation to achieve a common goal. This type of situated communication (Rickheit and Wachsmuth, 2006) has been somewhat addressed by robots and ECAs such as Max (Kopp et al, 2003;Kruijff et al, 2012), but generally the face-to-face paradigm appears to be more dominant in ECA research. Furthermore, even the environment in which Max is situated does not require extensive navigation, which is an activity commonly executed in sports.…”