Effects of voice-adaptation and social dialogue on perceptions of a robotic learning companion

Exploring the Effects of a Social Robot's Speech Entrainment and Backstory on Young Children's Emotion, Rapport, Relationship, and Learning

“…Thus, we suspect that the robot's social capabilities (such as nonverbal immediacy) can influence children's learning-as we have seen here and in multiple other studies discussed earlier-but that the influence of social behavior is moderated by other factors, such as the extent to which the robot's sociality is necessary for the learning activity to proceed smoothly (as in the case of conversation and storytelling-based activities), and the extent to which the robot's social behavior helps build rapport. This hypothesis is supported by Lubold and colleagues' recent work with middle school children and adults, in which a social robot with vocal entrainment contributed to increased learning on math tasks, though not increases in self-reported rapport (Lubold et al, 2016(Lubold et al, , 2018Lubold, 2017). Because the vocal entrainment served not only to match pitch and other vocal features, but also made the robot's text-to-speech voice much more expressive, these studies could not disentangle the effects of expressivity from entrainment-however, both expressivity and entrain increase the robot's social capabilities.…”

“…Some recent work has explored increasing prosodic synchrony in a speech-controlled child-robot game in order to promote cooperation and improve enjoyment (Chaspari and Lehman, 2016;Sadoughi et al, 2017). In addition, Lubold and colleagues developed several social voice-adaptive robots that adjust the pitch of the robot's text-to-speech voice to match that of its human interlocutor (Lubold et al, 2015(Lubold et al, , 2016(Lubold et al, , 2018Lubold, 2017). This vocal entrainment contributed to increased learning with undergraduate students as well as middle school students during math tasks, but did not increase self-reported rapport.…”

“…Then, since an individual may vary their pitch in each utterance, the pitch of each utterance was also shifted up or down slightly based on whether the child's most recent utterance was higher or lower. Unlike Lubold and colleagues (Lubold et al, 2016(Lubold et al, , 2018, we did not adapt the pitch contour of the robot's speech. Because the base sounds for the robot's speech were recorded by a human (not flat text-tospeech as in Lubold et al's work), the sounds had their own pitch contours.…”

“…Some prior work with adults provides evidence in support of this hypothesis (Kidd and Breazeal, 2008;Lubold et al, 2016Lubold et al, , 2018Lubold, 2017); however, there is little work yet exploring a social robot's rapport and relationship with young children. Thus, in this paper, we explored whether enabling a social robot to perform rapport-building behaviors, including speech and behavior entrainment, and giving the robot an appropriate backstory regarding its abilities, could help establish rapport and generate positive interactions with children, which we hypothesized could improve children's learning and engagement.…”

A Systematic Review of Adaptivity in Human-Robot Interaction

“…Recently, in 2016, Lubold [77] has conducted a study with a group of undergraduate students to measure the effect of voice-adaptation and social dialogue by a robotic learning companion on user's perception of robot's social presence. Their results showed that a social voice-adaptive dialogue has a significant effect on social presence as compared to a simple social dialogue.…”