Towards expressive musical robots: a cross-modal framework for emotional gesture, voice and music

Lim, Angelica; Ogata, Tetsuya; Okuno, Hiroshi G.

doi:10.1186/1687-4722-2012-3

Cited by 24 publications

(14 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, in [91], a four-dimensional model was developed for multimodal HRI to determine affect from voice, body movements, and music. The dimensions of the model were speed, intensity, regularity, and extent.…”

Section: Affect Models Used In Hrimentioning

confidence: 99%

A Survey of Autonomous Human Affect Detection Methods for Social Robots Engaged in Natural HRI

McColl

Hong

Hatakeyama

et al. 2015

J Intell Robot Syst

View full text Add to dashboard Cite

In Human-Robot Interactions (HRI), robots should be socially intelligent. They should be able to respond appropriately to human affective and social cues in order to effectively engage in bi-directional communications. Social intelligence would allow a robot to relate to, understand, and interact and share information with people in real-world humancentered environments. This survey paper presents an encompassing review of existing automated affect recognition and classification systems for social robots engaged in various HRI settings. Human-affect detection from facial expressions, body language, voice, and physiological signals are investigated, as well as from a combination of the aforementioned modes. The automated systems are described by their corresponding robotic and HRI applications, the sensors they employ, and the feature detection techniques and affect classification strategies utilized. This paper also discusses pertinent future research directions for promoting the development of socially intelligent robots capable of recognizing, classifying and responding to human affective states during real-time HRI.

show abstract

Section: Affect Models Used In Hrimentioning

confidence: 99%

A Survey of Autonomous Human Affect Detection Methods for Social Robots Engaged in Natural HRI

McColl

Hong

Hatakeyama

et al. 2015

J Intell Robot Syst

View full text Add to dashboard Cite

show abstract

“…xpression of emotional behaviors is an essential capability for social robots that interact with people in daily settings. Many robotics researchers have focused on developing robot hardware with enough degrees of freedoms (DOFs) to express various emotions by facial expressions [1][2][3][4], whole-body motions [5,6], and voice characteristics [7,8] as well as models that express appropriate emotions in interaction contexts [9,10]. These studies mainly used humanoid robots because human-like appearances and modalities are easily understandable for humans, but emotion expression models are not limited to such human-like robots.…”

Section: Introductionmentioning

confidence: 99%

Modeling the Timing and Duration of Grip Behavior to Express Emotions for a Social Robot

Zheng

Shiomi²,

Minato³

et al. 2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

This study addresses the effects of grip timing to express a robot's emotions to people while they are watching a video stimulus together. Past studies on human-robot touch interaction focused on the types of touch behaviors for expressing a robot's emotion, but timing factors have received less attention. In this study, we conducted data collection to investigate the appropriate grip timing to express heartwarming and horror emotions. Participants identified touch (grip) timing and durations by using a robot while they are watching video stimuli. Typically, they preferred a grip timing before a climax for horror videos but after a climax for heartwarming videos. We modeled the timing and durations by a fitting approach to probabilistic distribution to reproduce human-like touch behaviors, and also we implemented the models to an android robot.

show abstract

“…Since the dawn of robotics there has been an interest in making machines perform artistic and creative tasks in a human-like manner [1]. Music instrument playing, in particular, is an important challenge, because the skills necessary to play music from physical instruments are a1111111111 a1111111111 a1111111111 a1111111111 a1111111111…”

Section: Introductionmentioning

confidence: 99%

Gaussian process inference modelling of dynamic robot control for expressive piano playing

Scimeca¹,

Ng²,

Iida³

2020

PLoS ONE

View full text Add to dashboard Cite

Piano is a complex instrument, which humans learn to play after many years of practice. This paper investigates the complex dynamics of the embodied interactions between a human and piano, in order to gain insights into the nature of humans' physical dexterity and adaptability. In this context, the dynamic interactions become particularly crucial for delicate expressions, often present in advanced music pieces, which is the main focus of this paper. This paper hypothesises that the relationship between motor control for key-pressing and the generated sound is a manifold problem, with high-degrees of non-linearity in nature. We employ a minimalistic experimental platform based on a robotic arm equipped with a single elastic finger in order to systematically investigate the motor control and resulting outcome of piano sounds. The robot was programmed to run 3125 key-presses on a physical digital piano with varied control parameters. The obtained data was applied to a Gaussian Process (GP) inference modelling method, to train a network in terms of 10 playing styles, corresponding to different expressions generated by a Musical Instrument Digital Interface (MIDI). By analysing the robot control parameters and the output sounds, the relationship was confirmed to be highly nonlinear, especially when the rich expressions (such as a broad range of sound dynamics) were necessary. Furthermore this relationship was difficult and time consuming to learn with linear regression models, compared to the developed GPbased approach. The performance of the robot controller was also compared to that of an experienced human player. The analysis shows that the robot is able to generate sounds closer to humans' in some expressions, but requires additional investigations for others.

show abstract

Towards expressive musical robots: a cross-modal framework for emotional gesture, voice and music

Cited by 24 publications

References 27 publications

A Survey of Autonomous Human Affect Detection Methods for Social Robots Engaged in Natural HRI

A Survey of Autonomous Human Affect Detection Methods for Social Robots Engaged in Natural HRI

Modeling the Timing and Duration of Grip Behavior to Express Emotions for a Social Robot

Gaussian process inference modelling of dynamic robot control for expressive piano playing

Contact Info

Product

Resources

About