Neural network based reinforcement learning for audio–visual gaze control in human–robot interaction

Abstract

This paper introduces a novel neural network-based reinforcement learning approach for robot gaze control. Our approach enables a robot to learn and to adapt its gaze control strategy for human-robot interaction neither with the use of external sensors nor with human supervision. The robot learns to focus its attention onto groups of people from its own audio-visual experiences, independently of the number of people, of their positions and of their physical appearances. In particular, we use a recurrent neural network architecture in combination with Q-learning to find an optimal action-selection policy; we pre-train the network using a simulated environment that mimics realistic scenarios that involve speaking/silent participants, thus avoiding the need of tedious sessions of a robot interacting with people. Our experimental evaluation suggests that the proposed method is robust against parameter estimation, i.e. the parameter values yielded by the method do not have a decisive impact on the performance. The best results are obtained when both audio and visual information is jointly used. Experiments with the Nao robot indicate that our framework is a step forward towards the autonomous learning of socially acceptable gaze behavior.

Extended Gaze Following: Detecting Objects in Videos Beyond the Camera Field of View

“…Moreover, the availability of suitable datasets would ease future research on this topic. In parallel, we wish to use this framework in the future as a tool to improve the decision process of a robotic system in a social context such as [5].…”

Exploring Temporal Dependencies in Multimodal Referring Expressions with Mixed Reality

“…However, we want to move away from feature engineering and formulate our human-robot interaction scenario as a deep reinforcement learning problem. Recent studies in HRI showed impressive results in employing deep reinforcement learning for various applications [14,15,12]. The main challenge for deep learning approaches is the lack of training data from human studies but we plan to tackle this problem using our current Bayesian-based model to simulate human behaviour data as a prior for the deep reinforcement learning model.…”

Reinforcement Learning Approaches in Social Robotics

“…In deep networks, the selection of different hyper-parameters affects the accuracy of the algorithm [118]. This also applies to DRL, Lathuilière et al [86] presented several experiments to evaluate the impact of some of the principal parameters of their deep network structure. Thus far, model-free RL learning a value function or a policy through trial and error is the most commonly used approach in social robotics.…”