Reinforcement Learning Approaches in Social Robotics

Abstract

This article surveys reinforcement learning approaches in social robotics. Reinforcement learning is a framework for decision-making problems in which an agent interacts through trial-and-error with its environment to discover an optimal behavior. Since interaction is a key component in both reinforcement learning and social robotics, it can be a well-suited approach for real-world interactions with physically embodied social robots. The scope of the paper is focused particularly on studies that include social physical robots and real-world human-robot interactions with users. We present a thorough analysis of reinforcement learning approaches in social robotics. In addition to a survey, we categorize existent reinforcement learning approaches based on the used method and the design of the reward mechanisms. Moreover, since communication capability is a prominent feature of social robots, we discuss and group the papers based on the communication medium used for reward formulation. Considering the importance of designing the reward function, we also provide a categorization of the papers based on the nature of the reward. This categorization includes three major themes: interactive reinforcement learning, intrinsically motivated methods, and task performance-driven methods. The benefits and challenges of reinforcement learning in social robotics, evaluation methods of the papers regarding whether or not they use subjective and algorithmic measures, a discussion in the view of real-world reinforcement learning challenges and proposed solutions, the points that remain to be explored, including the approaches that have thus far received less attention is also given in the paper. Thus, this paper aims to become a starting point for researchers interested in using and applying reinforcement learning methods in this particular research field.

Reinforcement Learning for Autonomous Vehicle Movements in Wireless Sensor Networks

“…Social robotics environments [1] are very close to our own scenario. These robots are usually equipped with one or more sensors and move to improve data acquisition.…”

“…Then, the next speaker starts speaking from a new random position. 1 For the upcoming results (unless stated otherwise), we assume that v m ≥ 2v src . In case of the fixed set-up baseline, the microphones are placed on a uniform equidistant grid.…”

“…unmanned ground vehicles (e.g., social robots [1]) or unmanned aerial vehicles (UAV, e.g., drones [2]) -have a broad range of applications, including entertainment, industrial scenarios and ambientassisted living. They are usually considered as robots with interactive tasks, such as human-robot interaction or robot-robot collaboration.…”

Robot hand-eye cooperation based on improved inverse reinforcement learning

“…When experts complete a specific task, their decision is often optimal or near-optimal. It can be assumed that when the cumulative reward expectation generated by all strategies is less than that generated by expert strategies, the corresponding reward function is the reward function learned from the example (Akalin and Loutfi, 2021). Therefore, IRL can be defined as a reward function learned from expert examples.…”

SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents

“…requiring agents to both move and speak, and even to learn to interact in a diversity of pragmatic frames. Catalysing research on DRL and social skills seems even more relevant now that many application-oriented works are beginning to leverage RL and DRL into real world humanoid social robots [Akalin and Loutfi, 2021].…”