Establishing Human-Robot Trust through Music-Driven Robotic Emotion Prosody and Gesture

Savery, Richard; Rose, Ryan; Weinberg, Gil

doi:10.1109/ro-man46459.2019.8956386

Cited by 36 publications

(13 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Presentation includes the way the robot appears, such as changes based on different behavior ( Goetz et al, 2003 ). The final category, vocalics, includes concepts such as prosody ( Crumpton and Bethel, 2016 ), which have shown to improve trust and other human-robot interaction metrics ( Savery et al, 2019a ). The vast majority of these communication techniques require significant technical and financial expense and variation to a system, such as adding augmented reality technology or changing robot movements ( Saunderson and Nejat, 2019 ).…”

Section: Related Workmentioning

confidence: 99%

Before, Between, and After: Enriching Robot Communication Surrounding Collaborative Creative Activities

2021

Self Cite

View full text Add to dashboard Cite

Research in creative robotics continues to expand across all creative domains, including art, music and language. Creative robots are primarily designed to be task specific, with limited research into the implications of their design outside their core task. In the case of a musical robot, this includes when a human sees and interacts with the robot before and after the performance, as well as in between pieces. These non-musical interaction tasks such as the presence of a robot during musical equipment set up, play a key role in the human perception of the robot however have received only limited attention. In this paper, we describe a new audio system using emotional musical prosody, designed to match the creative process of a musical robot for use before, between and after musical performances. Our generation system relies on the creation of a custom dataset for musical prosody. This system is designed foremost to operate in real time and allow rapid generation and dialogue exchange between human and robot. For this reason, the system combines symbolic deep learning through a Conditional Convolution Variational Auto-encoder, with an emotion-tagged audio sampler. We then compare this to a SOTA text-to-speech system in our robotic platform, Shimon the marimba player.We conducted a between-groups study with 100 participants watching a musician interact for 30 s with Shimon. We were able to increase user ratings for the key creativity metrics; novelty and coherence, while maintaining ratings for expressivity across each implementation. Our results also indicated that by communicating in a form that relates to the robot’s core functionality, we can raise likeability and perceived intelligence, while not altering animacy or anthropomorphism. These findings indicate the variation that can occur in the perception of a robot based on interactions surrounding a performance, such as initial meetings and spaces between pieces, in addition to the core creative algorithms.

show abstract

Section: Related Workmentioning

confidence: 99%

Before, Between, and After: Enriching Robot Communication Surrounding Collaborative Creative Activities

2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…The authors considered the robot's role in its environment and adjusted its form and personality to meet human needs. Another meta-study of trust in HRI showed that the attributes of a robot have more influence on trust than environmental and human-related factors [49].…”

Section: The Methodological Gapmentioning

confidence: 99%

Exploring the Role of Trust and Expectations in CRI Using In-the-Wild Studies

et al. 2021

View full text Add to dashboard Cite

Studying interactions of children with humanoid robots in familiar spaces in natural contexts has become a key issue for social robotics. To fill this need, we conducted several Child–Robot Interaction (CRI) events with the Pepper robot in Polish and Japanese kindergartens. In this paper, we explore the role of trust and expectations towards the robot in determining the success of CRI. We present several observations from the video recordings of our CRI events and the transcripts of free-format question-answering sessions with the robot using the Wizard-of-Oz (WOZ) methodology. From these observations, we identify children’s behaviors that indicate trust (or lack thereof) towards the robot, e.g., challenging behavior of a robot or physical interactions with it. We also gather insights into children’s expectations, e.g., verifying expectations as a causal process and an agency or expectations concerning the robot’s relationships, preferences and physical and behavioral capabilities. Based on our experiences, we suggest some guidelines for designing more effective CRI scenarios. Finally, we argue for the effectiveness of in-the-wild methodologies for planning and executing qualitative CRI studies.

show abstract

“…From these studies, there are two works that present proposals for systems for automatic emotion detection (González & McMullen, 2020;Vinayagasundaram et al, 2016). Three other studies use techniques for generating affective or personalized music (Savery et al, 2019;Y.-C. Wu & Chen, 2016;Chen et al, 2020).…”

Section: Musical Composition Recommendation and Customizationmentioning

confidence: 99%

Affective computing in the context of music therapy: a systematic review

Santana

Lima

Torcate

et al. 2021

RSD

View full text Add to dashboard Cite

Music therapy is an effective tool to slow down the progress of dementia since interaction with music may evoke emotions that stimulates brain areas responsible for memory. This therapy is most successful when therapists provide adequate and personalized stimuli for each patient. This personalization is often hard. Thus, Artificial Intelligence (AI) methods may help in this task. This paper brings a systematic review of the literature in the field of affective computing in the context of music therapy. We particularly aim to assess AI methods to perform automatic emotion recognition applied to Human-Machine Musical Interfaces (HMMI). To perform the review, we conducted an automatic search in five of the main scientific databases on the fields of intelligent computing, engineering, and medicine. We search all papers released from 2016 and 2020, whose metadata, title or abstract contains the terms defined in the search string. The systematic review protocol resulted in the inclusion of 144 works from the 290 publications returned from the search. Through this review of the state-of-the-art, it was possible to list the current challenges in the automatic recognition of emotions. It was also possible to realize the potential of automatic emotion recognition to build non-invasive assistive solutions based on human-machine musical interfaces, as well as the artificial intelligence techniques in use in emotion recognition from multimodality data. Thus, machine learning for recognition of emotions from different data sources can be an important approach to optimize the clinical goals to be achieved through music therapy.

show abstract

Establishing Human-Robot Trust through Music-Driven Robotic Emotion Prosody and Gesture

Cited by 36 publications

References 46 publications

Before, Between, and After: Enriching Robot Communication Surrounding Collaborative Creative Activities

Before, Between, and After: Enriching Robot Communication Surrounding Collaborative Creative Activities

Exploring the Role of Trust and Expectations in CRI Using In-the-Wild Studies

Affective computing in the context of music therapy: a systematic review

Contact Info

Product

Resources

About