Systematic analysis of video data from different human–robot interaction studies: a categorization of social signals during error situations

Giuliani, Manuel; Mirnig, Nicole; Stollnberger, Gerald; Stadler, Susanne; Buchner, Roland; Tscheligi, Manfred

doi:10.3389/fpsyg.2015.00931

Cited by 56 publications

(53 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, recognizing social signals might help a robot to understand that an error happened. According to the frequencies of occurrence, gaze shifts and smile/ laughter carry most potential for error detection, which is in line with our previous findings in the study of Giuliani et al (2015). Upon a detailed analysis on the categories of social signals we found that people make significantly more gaze shifts during technical failures.…”

Section: Discussionsupporting

confidence: 90%

“…To base the user study on the previous findings from Giuliani et al (2015) and Mirnig et al (2015), we programmed the robot to commit two social norm violations and two technical failures in each session. Based on our previous research, we defined these two types of error as the typical mistakes robots make in HRI.…”

Section: Methodsmentioning

confidence: 99%

“…The induced errors were mainly modeled based on our previous findings on typical robot errors as reported in the studies of Giuliani et al (2015) and Mirnig et al (2015). Only LEGO task number 7 in the error condition was inspired by unusual requests as reported in the study of Salem et al (2015).…”

Section: User Study Designmentioning

confidence: 99%

See 2 more Smart Citations

To Err Is Robot: How Humans Assess and Act toward an Erroneous Social Robot

et al. 2017

Self Cite

View full text Add to dashboard Cite

We conducted a user study for which we purposefully programmed faulty behavior into a robot's routine. It was our aim to explore if participants rate the faulty robot different from an error-free robot and which reactions people show in interaction with a faulty robot. The study was based on our previous research on robot errors where we detected typical error situations and the resulting social signals of our participants during social human-robot interaction. In contrast to our previous work, where we studied video material in which robot errors occurred unintentionally, in the herein reported user study, we purposefully elicited robot errors to further explore the human interaction partners' social signals following a robot error. Our participants interacted with a human-like NAO, and the robot either performed faulty or free from error. First, the robot asked the participants a set of predefined questions and then it asked them to complete a couple of LEGO building tasks. After the interaction, we asked the participants to rate the robot's anthropomorphism, likability, and perceived intelligence. We also interviewed the participants on their opinion about the interaction. Additionally, we video-coded the social signals the participants showed during their interaction with the robot as well as the answers they provided the robot with. Our results show that participants liked the faulty robot significantly better than the robot that interacted flawlessly. We did not find significant differences in people's ratings of the robot's anthropomorphism and perceived intelligence. The qualitative data confirmed the questionnaire results in showing that although the participants recognized the robot's mistakes, they did not necessarily reject the erroneous robot. The annotations of the video data further showed that gaze shifts (e.g., from an object to the robot or vice versa) and laughter are typical reactions to unexpected robot behavior. In contrast to existing research, we assess dimensions of user experience that have not been considered so far and we analyze the reactions users express when a robot makes a mistake. Our results show that decoding a human's social signals can help the robot understand that there is an error and subsequently react accordingly.

show abstract

Section: Discussionsupporting

confidence: 90%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

To Err Is Robot: How Humans Assess and Act toward an Erroneous Social Robot

et al. 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…Ross (Ross et al, 2004 ) categorized system errors according to failure recoverability, defining anticipated errors (when the agent backtracks through the plan to achieve the same goal through an alternate course of action), exceptional errors (when the current plan cannot cope with the failure, and re-planning can be done to formulate a strategy to achieve the original goal) , unrecoverable errors (when the current plan cannot cope with the error and re-planning cannot be done), and socially recoverable errors (when the agent can continue on with the original plan with appropriate assistance from other agents within its environment). Giuliani et al ( 2015 ) classified failures according to their type, defining technical failures (caused by technical shortcomings of the robot) and social norm violations (when the robot deviates from the social script or uses inappropriate social signals, e.g., looking away from a person while talking to them).…”

Section: Defining and Classifying Errorsmentioning

confidence: 99%

Understanding and Resolving Failures in Human-Robot Interaction: Literature Review and Model Development

2018

View full text Add to dashboard Cite

While substantial effort has been invested in making robots more reliable, experience demonstrates that robots operating in unstructured environments are often challenged by frequent failures. Despite this, robots have not yet reached a level of design that allows effective management of faulty or unexpected behavior by untrained users. To understand why this may be the case, an in-depth literature review was done to explore when people perceive and resolve robot failures, how robots communicate failure, how failures influence people's perceptions and feelings toward robots, and how these effects can be mitigated. Fifty-two studies were identified relating to communicating failures and their causes, the influence of failures on human-robot interaction (HRI), and mitigating failures. Since little research has been done on these topics within the HRI community, insights from the fields of human computer interaction (HCI), human factors engineering, cognitive engineering and experimental psychology are presented and discussed. Based on the literature, we developed a model of information processing for robotic failures (Robot Failure Human Information Processing, RF-HIP), that guides the discussion of our findings. The model describes the way people perceive, process, and act on failures in human robot interaction. The model includes three main parts: (1) communicating failures, (2) perception and comprehension of failures, and (3) solving failures. Each part contains several stages, all influenced by contextual considerations and mitigation strategies. Several gaps in the literature have become evident as a result of this evaluation. More focus has been given to technical failures than interaction failures. Few studies focused on human errors, on communicating failures, or the cognitive, psychological, and social determinants that impact the design of mitigation strategies. By providing the stages of human information processing, RF-HIP can be used as a tool to promote the development of user-centered failure-handling strategies for HRIs.

show abstract

“…Nevertheless it is not surprising that human robot interactions might fail when at times even human-human interactions do. Giuliani et al [38], described two types of failures in HRI, i.e. social norm violations and technical failures.…”

Section: Social and Situation Awarenessmentioning

confidence: 99%

Socially Believable Robots

Moetesum¹,

Siddiqi²

2018

Human-Robot Interaction - Theory and Application

View full text Add to dashboard Cite

Long-term companionship, emotional attachment and realistic interaction with robots have always been the ultimate sign of technological advancement projected by sci-fi literature and entertainment industry. With the advent of artificial intelligence, we have indeed stepped into an era of socially believable robots or humanoids. Affective computing has enabled the deployment of emotional or social robots to a certain level in social settings like informatics, customer services and health care. Nevertheless, social believability of a robot is communicated through its physical embodiment and natural expressiveness. With each passing year, innovations in chemical and mechanical engineering have facilitated lifelike embodiments of robotics; however, still much work is required for developing a "social intelligence" in a robot in order to maintain the illusion of dealing with a real human being. This chapter is a collection of research studies on the modeling of complex autonomous systems. It will further shed light on how different social settings require different levels of social intelligence and what are the implications of integrating a socially and emotionally believable machine in a society driven by behaviors and actions.

show abstract

Systematic analysis of video data from different human–robot interaction studies: a categorization of social signals during error situations

Cited by 56 publications

References 27 publications

To Err Is Robot: How Humans Assess and Act toward an Erroneous Social Robot

To Err Is Robot: How Humans Assess and Act toward an Erroneous Social Robot

Understanding and Resolving Failures in Human-Robot Interaction: Literature Review and Model Development

Socially Believable Robots

Contact Info

Product

Resources

About