Getting to Know Your Robot Customers: Automated Analysis of User Identity and Demographics for Robots in the Wild

Abstract

Long-term studies with autonomous robots "in the wild" (deployed in real-world human-inhabited environments) are among the most laborious and resource-intensive endeavours in Human-Robot Interaction. Even if a robot system itself is robust and well-working, the analysis of the vast amounts of user data one aims to collect and analyse poses a significant challenge. This paper proposes an automated processing pipeline, using state-of-the-art computer-vision technology to estimate demographic factors from users' faces and re-identify them to establish usage patterns. It overcomes the problem of explicitly recruiting participants and having them fill questionnaires about their demographic background, and allows to study completely unsolicited and non-primed interactions over long periods of time. The paper offers a comprehensive assessment of the performance of the automated analysis with data from 68 days of continuous deployment of a robot in a care home, and also presents a set of findings obtained through the analysis, underpinning the viability of the approach.

Service robots in the hospitality industry: The case of Henn-na hotel, Japan

“…In addition to this aspect, the time for tasks to be performed by a robot is shorter and cheaper [119] and, for this reason, one of the greatest social concerns has to do with the risk of human unemployment caused by robots [13,121] and/or AI [24]. Other relevant social concerns are related to privacy [11,113], data protection [122] or ethical issues [119], which can hinder the implementation of social robots. For example, when robots are used 24 h a day, 7 days a week, in an environment for many weeks, they acquire a significant amount of data from users during social interaction, a situation that is difficult or almost impossible to avoid [122].…”

“…Other relevant social concerns are related to privacy [11,113], data protection [122] or ethical issues [119], which can hinder the implementation of social robots. For example, when robots are used 24 h a day, 7 days a week, in an environment for many weeks, they acquire a significant amount of data from users during social interaction, a situation that is difficult or almost impossible to avoid [122]. Aspects related to such concern may be a target for future research.…”

Taxonomy of Trust-Relevant Failures and Mitigation Strategies

“…Rich data sets, comprising task and error logs [32], user demographics [36], and navigation failures [15] have been obtained from these deployments, and analysed for the case study for this paper.…”

“…Given that the probability score is only intended to give an indication of the magnitude of a specific failure class, a BoE is most adequate for this assessment. The system data in [36] indicated that there were about 3.5 interactions per operational hour (i.e. time the robot is not resting or charging) with users that are actively using the robot.…”

Introduction to the Special Issue on AI for Long-Term Autonomy

“…Herrero et al [3] provide a comprehensive analysis of the user base of an autonomous system, which was deployed for long-term operation in a care home, hence studying long-term interaction of an autonomous robotic system. The core contributions of the paper are a processing pipeline to automatically estimate demographics (age, gender) of interacting users, and a model to discriminate between passive interactions (bystanders observing the mobile robot), and active interactions (users directly interacting with the robot).…”