Exploring embodiment and dueling bandit learning for preference adaptation in human-robot interaction

Schneider, Sebastian; Kummert, Franz

doi:10.1109/roman.2017.8172476

Cited by 5 publications

(10 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The past decade has seen a rapid growth of social robotics in diverse uncontrolled environments such as homes, schools, hospitals, shopping centers, or museums. In this review, we have seen various application domains in a range of fields including therapy [ 3 ], eldercare [ 62 ], entertainment [ 59 ], navigation [ 32 ], healthcare [ 44 ], education [ 58 ], personal robots [ 13 ], and rehabilitation [ 92 ]. Research in the field of social robotics and human-robot interaction becomes crucial as more and more robots are entering our lives.…”

Section: Evaluation Methodologiesmentioning

confidence: 99%

“…Bandit-based methods can be considered as a simplified case of RL in which the next state does not depend on the action taken by the agent. Different bandit-based methods explored in social robotics [ 4 , 44 , 45 , 46 , 47 ], such as dueling bandit learning [ 44 ], k-armed bandit method (multi-armed bandit) [ 4 , 45 , 46 ], and Exponential-Weight Algorithm for Exploration and Exploitation (Exp3) algorithm [ 47 ].…”

Section: Categorization Of Rl Approaches In Social Robotics Based mentioning

confidence: 99%

“…Learning user preferences to personalize the user experience is used in customizing advertisements and search results. A similar approach was applied in HRI studies [ 4 , 44 ]. Whereas the customization is done in the background for personalized experiences in websites using users’ clicks, it is adapted for social interactions by asking the user to select their preferences using the buttons.…”

Section: Categorization Of Rl Approaches In Social Robotics Based mentioning

confidence: 99%

“…In other words, these studies use a physical communication medium between the robot and the human. Schneider and Kummert [ 44 ] investigated a dueling bandit learning approach for preference learning. The algorithm draws two or more actions, and the relative preference is used as reward.…”

Section: Categorization Of Rl Approaches In Social Robotics Based mentioning

confidence: 99%

“…Unlike the explicit feedback category, in the implicit feedback category, the majority of studies include a social scenario such as robot tutoring, robots supporting the human in a game, etc. The studies under this category are [ 4 , 26 , 29 , 30 , 44 , 52 , 53 ].…”

Section: Categorization Of Rl Approaches In Social Robotics Based mentioning

confidence: 99%

See 4 more Smart Citations

Reinforcement Learning Approaches in Social Robotics

Akalın

Loutfi

2021

Sensors

View full text Add to dashboard Cite

This article surveys reinforcement learning approaches in social robotics. Reinforcement learning is a framework for decision-making problems in which an agent interacts through trial-and-error with its environment to discover an optimal behavior. Since interaction is a key component in both reinforcement learning and social robotics, it can be a well-suited approach for real-world interactions with physically embodied social robots. The scope of the paper is focused particularly on studies that include social physical robots and real-world human-robot interactions with users. We present a thorough analysis of reinforcement learning approaches in social robotics. In addition to a survey, we categorize existent reinforcement learning approaches based on the used method and the design of the reward mechanisms. Moreover, since communication capability is a prominent feature of social robots, we discuss and group the papers based on the communication medium used for reward formulation. Considering the importance of designing the reward function, we also provide a categorization of the papers based on the nature of the reward. This categorization includes three major themes: interactive reinforcement learning, intrinsically motivated methods, and task performance-driven methods. The benefits and challenges of reinforcement learning in social robotics, evaluation methods of the papers regarding whether or not they use subjective and algorithmic measures, a discussion in the view of real-world reinforcement learning challenges and proposed solutions, the points that remain to be explored, including the approaches that have thus far received less attention is also given in the paper. Thus, this paper aims to become a starting point for researchers interested in using and applying reinforcement learning methods in this particular research field.

show abstract

Section: Evaluation Methodologiesmentioning

confidence: 99%

Section: Categorization Of Rl Approaches In Social Robotics Based mentioning

confidence: 99%

Section: Categorization Of Rl Approaches In Social Robotics Based mentioning

confidence: 99%

Section: Categorization Of Rl Approaches In Social Robotics Based mentioning

confidence: 99%

Section: Categorization Of Rl Approaches In Social Robotics Based mentioning

confidence: 99%

See 3 more Smart Citations

Reinforcement Learning Approaches in Social Robotics

Akalın

Loutfi

2021

Sensors

View full text Add to dashboard Cite

show abstract

Comparing Robot and Human guided Personalization: Adaptive Exercise Robots are Perceived as more Competent and Trustworthy

Schneider

Kummert

2020

Int J of Soc Robotics

Self Cite

View full text Add to dashboard Cite

Learning and matching a user’s preference is an essential aspect of achieving a productive collaboration in long-term Human–Robot Interaction (HRI). However, there are different techniques on how to match the behavior of a robot to a user’s preference. The robot can be adaptable so that a user can change the robot’s behavior to one’s need, or the robot can be adaptive and autonomously tries to match its behavior to the user’s preference. Both types might decrease the gap between a user’s preference and the actual system behavior. However, the Level of Automation (LoA) of the robot is different between both methods. Either the user controls the interaction, or the robot is in control. We present a study on the effects of different LoAs of a Socially Assistive Robot (SAR) on a user’s evaluation of the system in an exercising scenario. We implemented an online preference learning system and a user-adaptable system. We conducted a between-subject design study (adaptable robot vs. adaptive robot) with 40 subjects and report our quantitative and qualitative results. The results show that users evaluate the adaptive robots as more competent, warm, and report a higher alliance. Moreover, this increased alliance is significantly mediated by the perceived competence of the system. This result provides empirical evidence for the relation between the LoA of a system, the user’s perceived competence of the system, and the perceived alliance with it. Additionally, we provide evidence for a proof-of-concept that the chosen preference learning method (i.e., Double Thompson Sampling (DTS)) is suitable for online HRI.

show abstract

Personalizing Activity Selection in Assistive Social Robots from Explicit and Implicit User Feedback

Maroto-Gómez,

Malfaz,

Castillo

et al. 2024

Int J of Soc Robotics

View full text Add to dashboard Cite

Robots in multi-user environments require adaptation to produce personalized interactions. In these scenarios, the user’s feedback leads the robots to learn from experiences and use this knowledge to generate adapted activities to the user’s preferences. However, preferences are user-specific and may suffer variations, so learning is required to personalize the robot’s actions to each user. Robots can obtain feedback in Human–Robot Interaction by asking users their opinion about the activity (explicit feedback) or estimating it from the interaction (implicit feedback). This paper presents a Reinforcement Learning framework for social robots to personalize activity selection using the preferences and feedback obtained from the users. This paper also studies the role of user feedback in learning, and it asks whether combining explicit and implicit user feedback produces better robot adaptive behavior than considering them separately. We evaluated the system with 24 participants in a long-term experiment where they were divided into three conditions: (i) adapting the activity selection using the explicit feedback that was obtained from asking the user how much they liked the activities; (ii) using the implicit feedback obtained from interaction metrics of each activity generated from the user’s actions; and (iii) combining explicit and implicit feedback. As we hypothesized, the results show that combining both feedback produces better adaptive values when correlating initial and final activity scores, overcoming the use of individual explicit and implicit feedback. We also found that the kind of user feedback does not affect the user’s engagement or the number of activities carried out during the experiment.

show abstract

Exploring embodiment and dueling bandit learning for preference adaptation in human-robot interaction

Cited by 5 publications

References 25 publications

Reinforcement Learning Approaches in Social Robotics

Reinforcement Learning Approaches in Social Robotics

Comparing Robot and Human guided Personalization: Adaptive Exercise Robots are Perceived as more Competent and Trustworthy

Personalizing Activity Selection in Assistive Social Robots from Explicit and Implicit User Feedback

Contact Info

Product

Resources

About