A Collaborative Human-Robot Game as a Test-bed for Modelling Multi-party, Situated Interaction

Skantze, Gabriel; Johansson, Martin; Beskow, Jonas

doi:10.1007/978-3-319-21996-7_37

Cited by 5 publications

(3 citation statements)

References 6 publications

(12 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Robotic heads (Mero, Furhat) instead focus on the importance of facial signals in communication (e.g., lip, eye and eyebrow movements to signal attention and emotions) and language learning (e.g., lip movements for pronunciation training). The Furhat robot, which is used in the present study, is uniquely anthropomorphic and expressive in comparison with other robots and has previously acted as e.g., host in competitive or collaborative quiz games (Skantze, Johansson, & Beskow, 2015), companion to autism children, detector of early signs of dementia (Jonell, Mendelson, & Storskog, 2017), or simulated Alzheimer patient (Kanov, 2017).…”

Section: Robot Typesmentioning

confidence: 99%

Interaction and collaboration in robot-assisted language learning for adults

Engwall

Lopes

2020

Computer Assisted Language Learning

View full text Add to dashboard Cite

This article analyses how robot-learner interaction in robotassisted language learning (RALL) is influenced by the interaction behaviour of the robot. Since the robot behaviour is to a large extent determined by the combination of teaching strategy, robot role and robot type, previous studies in RALL are first summarised with respect to which combinations that have been chosen, the rationale behind the choice and the effects on interaction and learning. The goal of the summary is to determine a suitable pedagogical setup for RALL with adult learners, since previous RALL studies have almost exclusively been performed with children and youths. A user study in which 33 adult second language learners practice Swedish in three-party conversations with an anthropomorphic robot head is then presented. It is demonstrated how different robot interaction behaviours influence interaction between the robot and the learners and between the two learners. Through an analysis of learner interaction, collaboration and learner ratings for the different robot behaviours, it is observed that the learners were most positive towards the robot behaviour that focused on interviewing one learner at the time (highest average ratings), but that they were the most active in sessions when the robot encouraged learner-learner interaction. Moreover, the preferences and activity differed between learner pairs, depending on, e.g., their proficiency level and how well they knew the peer. It is therefore concluded that the robot behaviour needs to adapt to such factors. In addition, collaboration with the peer played an important part in conversation practice sessions to deal with linguistic difficulties or communication problems with the robot.

show abstract

Section: Robot Typesmentioning

confidence: 99%

Interaction and collaboration in robot-assisted language learning for adults

Engwall

Lopes

2020

Computer Assisted Language Learning

View full text Add to dashboard Cite

show abstract

“…Based on these requirements, we decided to adapt the Card Game multi-party interaction setup (Skantze et al, 2015). The Card Game setup is a test-bed designed for studying single and multi-party interactions between a robot and human participants.…”

Section: Study: Affective Image Sorting Gamementioning

confidence: 99%

Real-time emotion generation in human-robot dialogue using large language models

Mishra,

Verdonschot,

Hagoort

et al. 2023

Front. Robot. AI

View full text Add to dashboard Cite

Affective behaviors enable social robots to not only establish better connections with humans but also serve as a tool for the robots to express their internal states. It has been well established that emotions are important to signal understanding in Human-Robot Interaction (HRI). This work aims to harness the power of Large Language Models (LLM) and proposes an approach to control the affective behavior of robots. By interpreting emotion appraisal as an Emotion Recognition in Conversation (ERC) tasks, we used GPT-3.5 to predict the emotion of a robot’s turn in real-time, using the dialogue history of the ongoing conversation. The robot signaled the predicted emotion using facial expressions. The model was evaluated in a within-subjects user study (N = 47) where the model-driven emotion generation was compared against conditions where the robot did not display any emotions and where it displayed incongruent emotions. The participants interacted with the robot by playing a card sorting game that was specifically designed to evoke emotions. The results indicated that the emotions were reliably generated by the LLM and the participants were able to perceive the robot’s emotions. It was found that the robot expressing congruent model-driven facial emotion expressions were perceived to be significantly more human-like, emotionally appropriate, and elicit a more positive impression. Participants also scored significantly better in the card sorting game when the robot displayed congruent facial expressions. From a technical perspective, the study shows that LLMs can be used to control the affective behavior of robots reliably in real-time. Additionally, our results could be used in devising novel human-robot interactions, making robots more effective in roles where emotional interaction is important, such as therapy, companionship, or customer service.

show abstract

“…Our GCS was tested with the Card Game scenario, which is a test-bed specifically designed for studying multi-party interactions involving joint attention to objects [30]. The Card Game setup consists of a Furhat robot [31], a touchscreen and up to two players, as seen in Fig.…”

Section: Test-bed: Card Gamementioning

confidence: 99%

Knowing Where to Look: A Planning-based Architecture to Automate the Gaze Behavior of Social Robots

Mishra¹,

Skantze

2022

2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)

Self Cite

View full text Add to dashboard Cite

Gaze cues play an important role in human communication and are used to coordinate turn-taking and joint attention, as well as to regulate intimacy. In order to have fluent conversations with people, social robots need to exhibit humanlike gaze behavior. Previous Gaze Control Systems (GCS) in HRI have automated robot gaze using data-driven or heuristic approaches. However, these systems tend to be mainly reactive in nature. Planning the robot gaze ahead of time could help in achieving more realistic gaze behavior and better eye-head coordination. In this paper, we propose and implement a novel planning-based GCS. We evaluate our system in a comparative within-subjects user study (N=26) between a reactive system and our proposed system. The results show that the users preferred the proposed system and that it was significantly more interpretable and better at regulating intimacy.

show abstract

A Collaborative Human-Robot Game as a Test-bed for Modelling Multi-party, Situated Interaction

Cited by 5 publications

References 6 publications

Interaction and collaboration in robot-assisted language learning for adults

Interaction and collaboration in robot-assisted language learning for adults

Real-time emotion generation in human-robot dialogue using large language models

Knowing Where to Look: A Planning-based Architecture to Automate the Gaze Behavior of Social Robots

Contact Info

Product

Resources

About