Teaching emotion expressions to a human companion robot using deep neural architectures

Churamani, Nikhil; Kerzel, Matthias; Strahl, Erik; Barros, Pablo; Wermter, Stefan

doi:10.1109/ijcnn.2017.7965911

Cited by 27 publications

(25 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These two significant works are thus related to work described here (projecting into a latent space), albeit our core affect state-space is a continuous one, and builds upon a more complex, nonlinear implementation model (deep GP). In other cases [77], [79], much like in machine learning approaches, a direct mapping via deep neural networks is pursued, from the latent space of affective expressions learnt in a bottom-up, feed-forward sweep to facial gestures, synthetic speech etc. Indeed the use of deep architectures, can lead to efficient implementation models capable to handle the multimodal nature of emotion [80].…”

Section: Discussionmentioning

confidence: 99%

Deep Construction of an Affective Latent Space via Multimodal Enactment

Boccignone

Conte

Cuculo

et al. 2018

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

We draw on a simulationist approach to the analysis of facially displayed emotions-e.g., in the course of a face-to-face interaction between an expresser and an observer. At the heart of such perspective lies the enactment of the perceived emotion in the observer. We propose a novel probabilistic framework based on a deep latent representation of a continuous affect space, which can be exploited for both the estimation and the enactment of affective states in a multimodal space (visible facial expressions and physiological signals). The rationale behind the approach lies in the large body of evidence from affective neuroscience showing that when we observe emotional facial expressions, we react with congruent facial mimicry. Further, in more complex situations, affect understanding is likely to rely on a comprehensive representation grounding the reconstruction of the state of the body associated with the displayed emotion. We show that our approach can address such problems in a unified and principled perspective, thus avoiding ad hoc heuristics while minimising learning efforts.

show abstract

Section: Discussionmentioning

confidence: 99%

Deep Construction of an Affective Latent Space via Multimodal Enactment

Boccignone

Conte

Cuculo

et al. 2018

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

show abstract

“…The NICO robot is a humanoid robot which serves as a fully controllable robot for multimodal human-robot interaction scenarios. NICO is an abbreviation for 'Neurally-Inspired Companion,' signifying its integration into assisting tasks like object grasping [11] and serving as a reliable platform for HRI studies including emotion recognition [6] and human-robot dialogue understanding [28] as well as for the assessment of critical design factors for robots, e.g., 'likeability' or safety issues [13], which are crucially important aspects for social robot research. The robot architecture and API are configured in a modular fashion to facilitate the integration, extension and combination of task-specific algorithms.…”

Section: Data Recording With the Nico Robotmentioning

confidence: 99%

Solving visual object ambiguities when pointing: an unsupervised learning approach

Jirak

Biertimpel

Kerzel

et al. 2020

Neural Comput & Applic

Self Cite

View full text Add to dashboard Cite

Whenever we are addressing a specific object or refer to a certain spatial location, we are using referential or deictic gestures usually accompanied by some verbal description. Particularly, pointing gestures are necessary to dissolve ambiguities in a scene and they are of crucial importance when verbal communication may fail due to environmental conditions or when two persons simply do not speak the same language. With the currently increasing advances of humanoid robots and their future integration in domestic domains, the development of gesture interfaces complementing human–robot interaction scenarios is of substantial interest. The implementation of an intuitive gesture scenario is still challenging because both the pointing intention and the corresponding object have to be correctly recognized in real time. The demand increases when considering pointing gestures in a cluttered environment, as is the case in households. Also, humans perform pointing in many different ways and those variations have to be captured. Research in this field often proposes a set of geometrical computations which do not scale well with the number of gestures and objects and use specific markers or a predefined set of pointing directions. In this paper, we propose an unsupervised learning approach to model the distribution of pointing gestures using a growing-when-required (GWR) network. We introduce an interaction scenario with a humanoid robot and define the so-called ambiguity classes. Our implementation for the hand and object detection is independent of any markers or skeleton models; thus, it can be easily reproduced. Our evaluation comparing a baseline computer vision approach with our GWR model shows that the pointing-object association is well learned even in cases of ambiguities resulting from close object proximity.

show abstract

“…It has a symmetrical and abstracted child-like appearance that aims to enable intuitive humanrobot interaction while avoiding the uncanny-valley effect. Behind the surface of the head, in the eyebrow and mouth area, a programmable LED display is placed, that can display basic emotions in the form of stylized facial expressions [14], [15]. The head features two 2 Megapixel sensors with a 70-degree field of vision.…”

Section: Affective Association Modellingmentioning

confidence: 99%

Disambiguating Affective Stimulus Associations for Robot Perception and Dialogue

Siqueira

Sutherland

Barros

et al. 2018

2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids)

Self Cite

View full text Add to dashboard Cite

Effectively recognising and applying emotions to interactions is a highly desirable trait for social robots. Implicitly understanding how subjects experience different kinds of actions and objects in the world is crucial for natural HRI interactions, with the possibility to perform positive actions and avoid negative actions. In this paper, we utilize the NICO robot's appearance and capabilities to give the NICO the ability to model a coherent affective association between a perceived auditory stimulus and a temporally asynchronous emotion expression. This is done by combining evaluations of emotional valence from vision and language. NICO uses this information to make decisions about when to extend conversations in order to accrue more affective information if the representation of the association is not coherent. Our primary contribution is providing a NICO robot with the ability to learn the affective associations between a perceived auditory stimulus and an emotional expression. NICO is able to do this for both individual subjects and specific stimuli, with the aid of an emotion-driven dialogue system that rectifies emotional expression incoherences. The robot is then able to use this information to determine a subject's enjoyment of perceived auditory stimuli in a real HRI scenario.

show abstract

Teaching emotion expressions to a human companion robot using deep neural architectures

Cited by 27 publications

References 17 publications

Deep Construction of an Affective Latent Space via Multimodal Enactment

Deep Construction of an Affective Latent Space via Multimodal Enactment

Solving visual object ambiguities when pointing: an unsupervised learning approach

Disambiguating Affective Stimulus Associations for Robot Perception and Dialogue

Contact Info

Product

Resources

About