Towards a platform-independent cooperative human-robot interaction system: I. Perception

Lallee, Stéphane; Lemaignan, Séverin; Lenz, Alexander; Melhuish, Chris; Natale, Lorenzo; Skachek, Sergey; Zant, Tijn van der; Warneken, Felix; Dominey, Peter Ford

doi:10.1109/iros.2010.5652697

Cited by 28 publications

(21 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This not only allows the extension of the robot's functionality by the addition and (re-)combination of new modules but also provides a mechanism to share higher-level functionality across several robotic platforms. This has been previously demonstrated [9], [11] by employing the cognitive elements of this architecture to control three different robots, namely iCUB [14], Jido 2 and BERT2 [15]. Here, we further demon- Fig.…”

Section: A Overviewmentioning

confidence: 71%

“…In particular, the ability of children to take a 'birdseye-view' and understand their and their partners actions as part of a shared plan [8] is a concept we have recently implemented in several robotic systems [9], [10], [11]. Similar work by Shah et al [12], has recently demonstrated that human-robot team performance is improved by shared planning in comparison to robots that are verbally commanded by the human.…”

Section: Contextmentioning

confidence: 99%

See 1 more Smart Citation

When shared plans go wrong: From atomic- to composite actions and back

Lenz

Lallee

Skachek

et al. 2012

2012 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

As elaborate human-robot interaction capabilities continue to develop, humans will increasingly be in proximity with robots, and the management of the ongoing control in case of breakdown becomes increasingly important: taking care of what happens when cooperation goes wrong. The current research addresses three categories of breakdowns where cooperation can go wrong. In the first category, the human detects some type of problem and generates a self-issued stop signal, with a physical palm up posture. In the second category, the human becomes distracted, and physically changes his orientation away from the shared space of cooperation. In the final category that we investigate, the human becomes physically close to the robot such that safety limits are reached and detected by the robot. In each of these three cases, the robot cognitive system detects the failure via the perception of distinct physical states from motion capture: the hand up posture; change in head orientation; and physical distance reaching a minimum threshold. In each case the robot immediately halts the current action. Then, the system should recover appropriately. Each error type returns a specific code, allowing the Supervisor system to handle the specific type of error. Our cognitive system allows the robot to learn composite actions, as a sequence of atomic actions. These composite actions can then be composed into higher level plans. When a plan fails at the level of a composite action, the recovery method is not trivial: should recovery take place at the level of the composite action, or the actual atomic action which physically failed? As the best recovery may depend on the physical context, we expand the plan into atomic actions, and recover at this level, allowing the user to specify whether the action should be skipped or retried. We demonstrate that this system allows graceful recovery from three principal categories of interaction breakdown, and provides an invaluable mechanism for preserving the integrity of cooperative HRI.

show abstract

Section: A Overviewmentioning

confidence: 71%

Section: Contextmentioning

confidence: 99%

When shared plans go wrong: From atomic- to composite actions and back

Lenz

Lallee

Skachek

et al. 2012

2012 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

show abstract

“…[11], [12]). Learning from demonstration under perceptual constraints was studied in [7], where it was shown that robots can learn more effectively when the perception of human demonstrators is restricted to be similar to their own.…”

Section: Related Workmentioning

confidence: 99%

Evaluating the effects of limited perception on interactive decisions in mixed robotic domains

Valtazanos¹,

Ramamoorthy²

2013

2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI)

View full text Add to dashboard Cite

Many robotic applications feature a mixture of interacting teleoperated and autonomous robots. In several such domains, human operators must make decisions using very limited perceptual information, e.g. by viewing only the noisy camera feed of their robot. There are many interaction scenarios where such restricted visibility impacts teleoperation performance, and where the role of autonomous robots needs to be reinforced. In this paper, we report on an experimental study assessing the effects of limited perception on human decision making, in interactions between autonomous and teleoperated NAO robots, where subjects do not have prior knowledge of how other agents will respond to their decisions. We evaluate the performance of several subjects under varying perceptual constraints in two scenarios; a simple cooperative task requiring collaboration with an autonomous robot, and a more demanding adversarial task, where an autonomous robot is actively trying to outperform the human. Our results indicate that limited perception has minimal impact on user performance when the task is simple. By contrast, when the other agent becomes more strategic, restricted visibility has an adverse effect on most subjects, with the performance level even falling below that achieved by an autonomous robot with identical restrictions. Our results could inform decisions about the division of control between humans and robots in mixedinitiative systems, and in determining when autonomous robots should intervene to assist operators.

show abstract

“…Research in human social interaction has demonstrated that one of the unique abilities that provides the basis for human social interaction is cooperation [2]. Inspired by this, we have developed a methodology for cooperative human-robot interaction systems with language playing an important role [3][4][5][6][7]. We believe that if humanoid robots are to engage with humans in useful, timely and cooperative activities, they must be able to learn from their experience with humans, and importantly, to share their knowledge in a suitable way.…”

Section: Introductionmentioning

confidence: 99%

Proof of concept for a user-centered system for sharing cooperative plan knowledge over extended periods and crew changes in space-flight operations

Sorce

Pointeau

Petit

et al. 2015

2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)

View full text Add to dashboard Cite

With the Robonaut-2 humanoid robot now permanently flying on the ISS, the potential role for robots participating in cooperative activity in space is becoming a reality. Recent research has demonstrated that cooperation in the joint achievement of shared goals is a promising framework for human interaction with robots, with application in space. Perhaps more importantly, with the turn-over of crew members, robots could play an important role in maintaining and transferring expertise between outgoing and incoming crews. In this context, the current research builds on our experience in systems for cooperative human-robot interaction, introducing novel interface and interaction modalities that exploit the long-term experience of the robot. We implement a system where the human agent can teach the Nao humanoid new actions by physical demonstration, visual imitation, and spoken command. These actions can then be composed into joint action plans that coordinate the cooperation between agent and human.We also implement algorithms for an Autobiographical Memory (ABM) that provides access to of all of the robots interaction experience.These functions are assembled in a novel interaction paradigm for the capture, maintenance and transfer of knowledge in a five-tiered structure. The five tiers allow the robot to 1) learn simple behaviors, 2) learn shared plans composed from the learned behaviors, 3) execute the learned shared plans efficiently, 4) teach shared plans to new humans, and 5) answer questions from the human to better understand the origin of the shared plan. Our results demonstrate the feasibility of this system and indicate that such humanoid robot systems will provide a potential mechanism for the accumulation and transfer of knowledge, between humans who are not co-present. Applications to space flight operations as a target scenario are discussed.Index Terms-human-robot interaction, shared-plan, behavior learning, robotic teaching, space-flight operations.

show abstract

Towards a platform-independent cooperative human-robot interaction system: I. Perception

Cited by 28 publications

References 28 publications

When shared plans go wrong: From atomic- to composite actions and back

When shared plans go wrong: From atomic- to composite actions and back

Evaluating the effects of limited perception on interactive decisions in mixed robotic domains

Proof of concept for a user-centered system for sharing cooperative plan knowledge over extended periods and crew changes in space-flight operations

Contact Info

Product

Resources

About