A friendly gesture: Investigating the effect of multimodal robot behavior in human-robot interaction

Salem, Maha; Rohlfing, Katharina J.; Kopp, Stefan; Joublin, Frank

doi:10.1109/roman.2011.6005285

Cited by 78 publications

(42 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…• In Condition 3, the incongruent multimodal (speechgesture) condition, the robot presented the participant pantomimic gesture conveying the act of opening the cupboard; deictic gesture pointing at designated position [37] with the identical set of nine verbal instructions used in condition 1. Again, in addition, they were accompanied by a total of 21 gestures, out of which ten gestures (47.6%) semantically matched the verbal instruction, while the remaining eleven gestures (52.4%) were semantically non-matching, e.g., the robot occasionally said "put it up there" but pointed downwards.…”

Section: Conditionsmentioning

confidence: 99%

Generation and Evaluation of Communicative Robot Gesture

Salem¹,

Kopp

Wachsmuth

et al. 2012

Int J of Soc Robotics

Self Cite

137

View full text Add to dashboard Cite

How is communicative gesture behavior in robots perceived by humans? Although gesture is crucial in social interaction, this research question is still largely unexplored in the field of social robotics. Thus, the main objective of the present work is to investigate how gestural machine behaviors can be used to design more natural communication in social robots. The chosen approach is twofold. Firstly, the technical challenges encountered when implementing a speech-gesture generation model on a robotic platform are tackled. We present a framework that enables the humanoid robot to flexibly produce synthetic speech and co-verbal hand and arm gestures at run-time, while not being limited to a predefined repertoire of motor actions. Secondly, the achieved flexibility in robot gesture is exploited in controlled experiments. To gain a deeper understanding of how communicative robot gesture might impact and shape human perception and evaluation of human-robot interaction, we conducted a between-subjects experimental study using the humanoid robot in a joint task scenario. We manipulated the non-verbal behaviors of the robot in three experimental conditions, so that it would refer to objects by utilizing either (1) unimodal (i.e., speech only) utterances, (2) congruent multimodal (i.e., semantically matching speech and gesture) or (3) incongruent multimodal (i.e., semantically non-matching speech and gesture) utterances. Our findings reveal that the robot is evaluated more positively when nonverbal behaviors such as hand and arm gestures are displayed along with speech, even if they do not semantically match the spoken utterance.

show abstract

Section: Conditionsmentioning

confidence: 99%

Generation and Evaluation of Communicative Robot Gesture

Salem¹,

Kopp

Wachsmuth

et al. 2012

Int J of Soc Robotics

Self Cite

137

View full text Add to dashboard Cite

show abstract

“…Along with speech, non-verbal behavior plays an important role in communication and interaction. Numerous researchers have been working on developing gestures for agents [1][2][3] and robots [4,5]. However, how an agent/robot can most effectively use gestures to interact with people is not well studied.…”

Section: Introductionmentioning

confidence: 99%

“…The agent was rated as more natural, warmhearted, agile and committed when presenting self-touching gestures. Salem et al [5] found that a robot is evaluated more positively when non-verbal behaviors such as hand and arm gestures accompany speech. Neff et al [9,10] conducted experiments to understand how the Big Five traits of emotional stability and extraversion correlate with changes in verbal and nonverbal behavior.…”

Section: Introductionmentioning

confidence: 99%

An Examination of Whether People Prefer Agents Whose Gestures Mimic Their Own

Luo

Ng-Thow-Hing

Neff

2013

Intelligent Virtual Agents

View full text Add to dashboard Cite

Abstract. Do people prefer gestures that are similar to their own? There is evidence that in conversation, people will tend to adopt the postures, gestures and mannerisms of their interaction partners [1]. This mirroring, sometimes called the "chameleon effect", is associated with affiliation, rapport and liking. It may be that a useful way to build rapport in human-agent/robot interaction is to have the agent/robot perform gestures similar to the human. As a step towards that, this study explores if people prefer gestures similar to their own over gestures similar to those of other people. Participants were asked to evaluate a series of agent motions, some of which mimic their own gestures, and rate their preference. A second study first showed participants videos of their own gesturing to see if self-awareness would impact their preference. Different scenarios for soliciting gesture behavior were also explored. Evidence suggests people do have some preference for motions similar to their own, but self-awareness has no effect.

show abstract

“…Our tendency to make these subjective judgments makes it crucial that all aspects of a robot's appearance, from the broad shape of the body to the subtle tilt of the eyes, are refined until they convey the image desired. In addition to appearance, the motion of the robot is critical to human perception; motion parameters such as acceleration and curvature [Saerbeck and Bartneck 2010], music synchrony [Avrunin et al 2011], inclusion of gestures [Salem et al 2011] or expressions [Blow et al 2006], and even unintended cues like motor noise [Hegel et al 2011] all affect a human's perception of the robot. Mori postulated that our acceptance of a robot form increases with its increasing similarity to humans, up to a point at which even slight deviations cause a sense of uncanniness [Mori 1970].…”

Section: Chaptermentioning

confidence: 99%

Sensing through structure

Slyper

Poupyrev

Hodgins

2010

Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction

View full text Add to dashboard Cite

We present an approach to designing input devices that focuses on the structure of materials. We explore and visualize how a material reacts under manipulation, and harness the material's properties to design new movement sensors. Two benefits spring out of this approach. One, simpler sensing emerges from making use of existing structure in the material. Two, by working with the natural structure of the material, we create input devices with readily recognizable affordances. We present six projects using this approach. We use the natural structure (coordination) of the human body to enable a mapping from five clothing-mounted accelerometers to high-quality motion capture data, creating a low-cost performance animation system. We design silicone input devices with embedded texture allowing single-camera tracking. We study squishable, conformable materials such as foam and silicone, and create a vocabulary of unit structures (shaped cuts in the material) for harnessing patterns of compression/tension to capture particular manipulations. We use this vocabulary to build soft sensing skeletons for stuffed animals, making foam cores with e-textile versions of our unit structures. We also use this vocabulary to design a tongue input device for a collaboration with Disney Imagineering. Finally, we rethink this vocabulary and apply it to capturing, using air pressure sensors, manipulations of hollow 3D-printed rubber shapes, and 3D-print several interactive robots incorporating the new vocabulary.

show abstract

A friendly gesture: Investigating the effect of multimodal robot behavior in human-robot interaction

Cited by 78 publications

References 15 publications

Generation and Evaluation of Communicative Robot Gesture

Generation and Evaluation of Communicative Robot Gesture

An Examination of Whether People Prefer Agents Whose Gestures Mimic Their Own

Sensing through structure

Contact Info

Product

Resources

About