Generation and Evaluation of Communicative Robot Gesture

Abstract

How is communicative gesture behavior in robots perceived by humans? Although gesture is crucial in social interaction, this research question is still largely unexplored in the field of social robotics. Thus, the main objective of the present work is to investigate how gestural machine behaviors can be used to design more natural communication in social robots. The chosen approach is twofold. Firstly, the technical challenges encountered when implementing a speech-gesture generation model on a robotic platform are tackled. We present a framework that enables the humanoid robot to flexibly produce synthetic speech and co-verbal hand and arm gestures at run-time, while not being limited to a predefined repertoire of motor actions. Secondly, the achieved flexibility in robot gesture is exploited in controlled experiments. To gain a deeper understanding of how communicative robot gesture might impact and shape human perception and evaluation of human-robot interaction, we conducted a between-subjects experimental study using the humanoid robot in a joint task scenario. We manipulated the non-verbal behaviors of the robot in three experimental conditions, so that it would refer to objects by utilizing either (1) unimodal (i.e., speech only) utterances, (2) congruent multimodal (i.e., semantically matching speech and gesture) or (3) incongruent multimodal (i.e., semantically non-matching speech and gesture) utterances. Our findings reveal that the robot is evaluated more positively when nonverbal behaviors such as hand and arm gestures are displayed along with speech, even if they do not semantically match the spoken utterance.

Learning-based modeling of multimodal behaviors for humanlike robots

“…These results are consistent with previous work on robots' use of gestures to support speech, which found no differences in participants' ratings of a robot displaying gestures that were semantically incongruent with its speech and one that displayed congruent behaviors in measures of how lively, active, engaged, communicative, and fun-loving they perceived the robot to be [34].…”

“…These examples highlight the importance of temporal alignment among different modalities of behavior to improve human-robot collaboration, perceptions of the robot, and overall user experience. Researchers have also developed models of gesture to improve human-robot interaction, including a probabilistic model to generate the four common types of gestures that are aligned with speech and achieve varying levels of expressiveness for robots [31] and a method for aligning speech and gesture strokes and smoothing transitions between gestures to produce more fluent behavior [34]. Robot that appropriately align their speech and gesture using such models were found to be more natural [32].…”

Modeling and Evaluating Narrative Gestures for Humanlike Robots

“…This line of research also includes novel approaches to and control architectures for generating gestures [4,23,30]. For instance, Salem et al [30] developed a control architecture for deictic and iconic gestures for a humanoid robot performing in a human-robot joint task. Similarly, Bremner et al [4] introduced a gesture production approach based on actuator end-point and trajectory to generate open-hand gestures.…”

“…This thread of research has shown that gestures positively shape participants' affective states [22], behavioral responses to the gestures [28], engagement in the interaction [5], and perceptions of the robot when gestures and speech are mismatched [30]. Such research involved laboratory studies in which participants performed a task with a physical robot [5,30] or video-based studies in which participants observed robots performing gestures [22,23,28].…”

“…What gestures are most effective in achieving improvements in outcomes like user task performance and perceptions of the robot? Research in human-robot interaction has explored how robots might use gestures to give directions [25], to coordinate joint activities [30], and to persuade their users [7]. While these studies demonstrate the potential robot gestures hold in shaping humanrobot interaction, research to date has not explored how robots might selectively use different types of gestures to target improvements in specific interaction outcomes such as user task performance, perceptions of the robot's effectiveness, and perceptions of the social and affective characteristics of the interaction.…”