Joint action perception to enable fluent human-robot teamwork

Iqbal, Tarıq; Gonzales, Michael J.; Riek, Laurel D.

doi:10.1109/roman.2015.7333671

Cited by 21 publications

(7 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second, in a public setting, it is not enough for a robot simply to achieve its task-based goals; instead, it must also be able to satisfy the social goals and obligations that arise through interactions with people in real-world settings. Therefore, we argue that task-based, social interaction in a public space can be seen as an instance of multimodal joint action [32,58].…”

Section: Introductionmentioning

confidence: 99%

Automatically Classifying User Engagement for Dynamic Multi-party Human–Robot Interaction

Foster

Gaschler

Giuliani

2017

Int J of Soc Robotics

View full text Add to dashboard Cite

A robot agent designed to engage in real-world human-robot joint action must be able to understand the social states of the human users it interacts with in order to behave appropriately. In particular, in a dynamic public space, a crucial task for the robot is to determine the needs and intentions of all of the people in the scene, so that it only interacts with people who intend to interact with it. We address the task of estimating the engagement state of customers for a robot bartender based on the data from audiovisual sensors. We begin with an offline experiment using hidden Markov models, confirming that the sensor data contains the information necessary to estimate user state. We then present two strategies for online state estimation: a rule-based classifier based on observed human behaviour in real bars, and a set of supervised classifiers trained on a labelled corpus. These strategies are compared in offline cross-validation, in an online user study, and through validation against a separate test corpus. These studies show that while the trained classifiers are best in a cross-validation setting, the rule-based classifier performs best with novel data; however, all classiThis article integrates and extends the work described in the following conference papers: [17,20,23,24 Bristol Robotics Laboratory, University of the West of England, Bristol, UK fiers also change their estimate too frequently for practical use. To address this issue, we present a final classifier based on Conditional Random Fields: this model has comparable performance on the test data, with increased stability. In summary, though, the rule-based classifier shows competitive performance with the trained classifiers, suggesting that for this task, such a simple model could actually be a preferred option, providing useful online performance while avoiding the implementation and data-scarcity issues involved in using machine learning for this task.

show abstract

Section: Introductionmentioning

confidence: 99%

Automatically Classifying User Engagement for Dynamic Multi-party Human–Robot Interaction

Foster

Gaschler

Giuliani

2017

Int J of Soc Robotics

View full text Add to dashboard Cite

show abstract

“…To adapt this approach for human subjects, we used learning techniques to cover quickly a large parameter range across four different conditions over an order of magnitude in visual duration. Similar expansive experiments for mapping multiple conditions and parameters could be used to explore the efficacy of machine-human interfaces [43,44], determine parameters for athletic performance and diagnose motor or cognitive conditions with characteristic dynamics [45]. In the context of this study, this combination of techniques has been used to illustrate how visual perception can be augmented with audio signals to enhance coordination.…”

Section: Discussionmentioning

confidence: 99%

Audio cues enhance mirroring of arm motion when visual cues are scarce

Lee

Esposito

Cohen

2019

J. R. Soc. Interface.

View full text Add to dashboard Cite

Swing in a crew boat, a good jazz riff, a fluid conversation: these tasks require extracting sensory information about how others flow in order to mimic and respond. To determine what factors influence coordination, we build an environment to manipulate incoming sensory information by combining virtual reality and motion capture. We study how people mirror the motion of a human avatar’s arm as we occlude the avatar. We efficiently map the transition from successful mirroring to failure using Gaussian process regression. Then, we determine the change in behaviour when we introduce audio cues with a frequency proportional to the speed of the avatar’s hand or train individuals with a practice session. Remarkably, audio cues extend the range of successful mirroring to regimes where visual information is sparse. Such cues could facilitate joint coordination when navigating visually occluded environments, improve reaction speed in human–computer interfaces or measure altered physiological states and disease.

show abstract

“…For example, our own recent work uses non-linear analysis techniques and motion modeling to explore how teams synchronize with one another during psychomotor entrainment tasks (Iqbal & Riek, 2014Iqbal, Gonzales, and Riek, 2015;Rack, Iqbal, and Riek, 2015). As synchronizing with others is a key indicator of brain health and social development, the methods we are developing may be useful for therapists and researchers working with individuals with autism, post-traumatic stress disorder, and traumatic brain injury.…”

Section: Diagnosing and Studying Schizophreniamentioning

confidence: 99%

Robotics Technology in Mental Health Care

Riek

2016

Artificial Intelligence in Behavioral and Mental Health Care

Self Cite

View full text Add to dashboard Cite

This chapter discusses the existing and future use of robotics and intelligent sensing technology in mental health care. While the use of this technology is nascent in mental health care, it represents a potentially useful tool in the practitioner's toolbox. The goal of this chapter is to provide a brief overview of the field, discuss the recent use of robotics technology in mental health care practice, explore some of the design issues and ethical issues of using robots in this space, and finally to explore the potential of emerging technology

show abstract

Joint action perception to enable fluent human-robot teamwork

Cited by 21 publications

References 39 publications

Automatically Classifying User Engagement for Dynamic Multi-party Human–Robot Interaction

Automatically Classifying User Engagement for Dynamic Multi-party Human–Robot Interaction

Audio cues enhance mirroring of arm motion when visual cues are scarce

Robotics Technology in Mental Health Care

Contact Info

Product

Resources

About