Continual Learning for Affective Robotics: Why, What and How?

Churamani, Nikhil; Kalkan, Sinan; Güneş, Hatice

doi:10.1109/ro-man47096.2020.9223564

Cited by 48 publications

(28 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another line of attack could be to learn from users during deployment (similar to Zhao and Eskenazi, 2016 ; Hancock et al, 2019 ; Sreedhar et al, 2020 ; Liu, 2020 ; Irfan et al, 2020b ), where the feedback could be the user response or emotions of the user to evaluate the user satisfaction with the agent’s responses. Adapting to the emotions of the user could improve the naturalness of the agent, and provide an additional level of personalisation within therapy, education or entertainment ( Castellano et al, 2010 ; Irfan et al, 2020b ; Churamani et al, 2020 ).…”

Section: Discussionmentioning

confidence: 99%

Coffee With a Hint of Data: Towards Using Data-Driven Approaches in Personalised Long-Term Interactions

Irfan

Hellou²,

Belpaeme

2021

Front. Robot. AI

View full text Add to dashboard Cite

While earlier research in human-robot interaction pre-dominantly uses rule-based architectures for natural language interaction, these approaches are not flexible enough for long-term interactions in the real world due to the large variation in user utterances. In contrast, data-driven approaches map the user input to the agent output directly, hence, provide more flexibility with these variations without requiring any set of rules. However, data-driven approaches are generally applied to single dialogue exchanges with a user and do not build up a memory over long-term conversation with different users, whereas long-term interactions require remembering users and their preferences incrementally and continuously and recalling previous interactions with users to adapt and personalise the interactions, known as the lifelong learning problem. In addition, it is desirable to learn user preferences from a few samples of interactions (i.e., few-shot learning). These are known to be challenging problems in machine learning, while they are trivial for rule-based approaches, creating a trade-off between flexibility and robustness. Correspondingly, in this work, we present the text-based Barista Datasets generated to evaluate the potential of data-driven approaches in generic and personalised long-term human-robot interactions with simulated real-world problems, such as recognition errors, incorrect recalls and changes to the user preferences. Based on these datasets, we explore the performance and the underlying inaccuracies of the state-of-the-art data-driven dialogue models that are strong baselines in other domains of personalisation in single interactions, namely Supervised Embeddings, Sequence-to-Sequence, End-to-End Memory Network, Key-Value Memory Network, and Generative Profile Memory Network. The experiments show that while data-driven approaches are suitable for generic task-oriented dialogue and real-time interactions, no model performs sufficiently well to be deployed in personalised long-term interactions in the real world, because of their inability to learn and use new identities, and their poor performance in recalling user-related data.

show abstract

Section: Discussionmentioning

confidence: 99%

Coffee With a Hint of Data: Towards Using Data-Driven Approaches in Personalised Long-Term Interactions

Irfan

Hellou²,

Belpaeme

2021

Front. Robot. AI

View full text Add to dashboard Cite

show abstract

“…Continual learning is essential for robotics since robots interacting with the environment and the humans continuously discover new tasks, contexts and interactions. For a widespread use of robots, whenever needed, robots are expected to learn new tasks and skills, and to adapt to new experiences or contexts ( Feng et al, 2019 ; Churamani et al, 2020 ; Kasaei et al, 2021 ; Ugur and Oztop, 2021 ).…”

Section: Related Workmentioning

confidence: 99%

“…There has been substantial work lately on addressing lifelong learning in robots, to enable lifelong learning in various robot capabilities, ranging from perception to navigation and manipulation (for reviews, see Churamani et al, 2020 ; Ugur and Oztop 2021 ; Feng et al, 2019 ; Kasaei et al, 2021 ). For example, Feng et al (2019) benchmarked existing continual learning strategies for object recognition for a robot interacting continually with the environment.…”

Section: Related Workmentioning

confidence: 99%

Mind Your Manners! A Dataset and a Continual Learning Approach for Assessing Social Appropriateness of Robot Actions

2022

Self Cite

View full text Add to dashboard Cite

To date, endowing robots with an ability to assess social appropriateness of their actions has not been possible. This has been mainly due to (i) the lack of relevant and labelled data and (ii) the lack of formulations of this as a lifelong learning problem. In this paper, we address these two issues. We first introduce the Socially Appropriate Domestic Robot Actions dataset (MANNERS-DB), which contains appropriateness labels of robot actions annotated by humans. Secondly, we train and evaluate a baseline Multi Layer Perceptron and a Bayesian Neural Network (BNN) that estimate social appropriateness of actions in MANNERS-DB. Finally, we formulate learning social appropriateness of actions as a continual learning problem using the uncertainty of Bayesian Neural Network parameters. The experimental results show that the social appropriateness of robot actions can be predicted with a satisfactory level of precision. To facilitate reproducibility and further progress in this area, MANNERS-DB, the trained models and the relevant code are made publicly available at https://github.com/jonastjoms/MANNERS-DB.

show abstract

“…Concerning continual learning in HRI, Churamani et al [42] discuss its importance for creating fully adaptive affective robots and how to utilize it for perception and behavior learning with adaptation. In [43], a CNN classifier for object detection was enriched with incremental learning capabilities to add new object classes for classification, while in [44], adaptive incremental learning through interaction of social robots with humans was proposed.…”

Section: Related Workmentioning

confidence: 99%

Visual Robotic Perception System with Incremental Learning for Child–Robot Interaction Scenarios

et al. 2021

View full text Add to dashboard Cite

This paper proposes a novel lightweight visual perception system with Incremental Learning (IL), tailored to child–robot interaction scenarios. Specifically, this encompasses both an action and emotion recognition module, with the former wrapped around an IL system, allowing novel actions to be easily added. This IL system enables the tutor aspiring to use robotic agents in interaction scenarios to further customize the system according to children’s needs. We perform extensive evaluations of the developed modules, achieving state-of-the-art results on both the children’s action BabyRobot dataset and the children’s emotion EmoReact dataset. Finally, we demonstrate the robustness and effectiveness of the IL system for action recognition by conducting a thorough experimental analysis for various conditions and parameters.

show abstract

Continual Learning for Affective Robotics: Why, What and How?

Cited by 48 publications

References 66 publications

Coffee With a Hint of Data: Towards Using Data-Driven Approaches in Personalised Long-Term Interactions

Coffee With a Hint of Data: Towards Using Data-Driven Approaches in Personalised Long-Term Interactions

Mind Your Manners! A Dataset and a Continual Learning Approach for Assessing Social Appropriateness of Robot Actions

Visual Robotic Perception System with Incremental Learning for Child–Robot Interaction Scenarios

Contact Info

Product

Resources

About