Autonomously Learning One-To-Many Social Interaction Logic from Human-Human Interaction Data

Abstract

We envision a future where service robots autonomously learn how to interact with humans directly from human-human interaction data, without any manual intervention. In this paper, we present a data-driven pipeline that: (1) takes in low-level data of a human shopkeeper interacting with multiple customers (28 hours of collected data); (2) autonomously extracts high-level actions from that data; and (3) learns-without manual intervention-how a robotic shopkeeper should respond to customers' actions online. Our proposed system for learning the interaction logic uses neural networks to first learn which customer actions are important to respond to and then learn how the shopkeeper should respond to those important customer actions. We present a novel technique for learning which customer actions are important by first learning the hidden causal relationship between customer and shopkeeper actions. In an offline evaluation, we show that our proposed technique significantly outperforms state-of-the-art baselines, in both which customer actions are important and how to respond to them. CCS CONCEPTS • Computing methodologies → Learning from demonstrations; • Human-centered computing → HCI theory, concepts and models.

Reproducibility in Human-Robot Interaction: Furthering the Science of HRI

“…Examples include video studies whose authors included their experimental stimuli as supplemental materials to their publications [ 14 , 15 ]. Additionally, at least one paper’s authors publicly released their full dataset and a software specification of the neural network architecture used in their publication [ 16 ].…”

Evaluating Automatic Body Orientation Detection for Indoor Location from Skeleton Tracking Data to Detect Socially Occupied Spaces Using the Kinect v2, Azure Kinect and Zed 2i

“…Approaches using commercial depth cameras include the Kinect v2 camera in an egocentric perspective in robots for conversational participation and events, limiting the analysis to static scenarios evaluating only the physically occupied space by interacting with the artificial participants [ 35 , 36 ]. However, this use demonstrates their great potential in acquiring trajectory and relevant social features due to the processed skeleton data, easiness of installation, and low costs without storing video data from the scene, allowing researchers to exploit these data to extract human behaviour [ 37 , 38 , 39 ]. Additional depth cameras available for the public include the Azure Kinect, the successor of the Kinect v2, mainly used for industry and healthcare with promising human activity detection [ 40 ].…”

A Review on HRI for Child Learning

“…Oudah et al [24] used repeated stochastic game is proposed for online learning, is effective way to interact with the people. An another Machine learning algorithm followed in [25] to predict the shopkeeper reaction from interaction of customer, Attention network utilizes the neural network to learn firstly and the Interaction network provide the interaction of shopkeeper. The principal component analysis based learning and engaging application of child is described by [26].…”