The Robot Management System: A Framework for Conducting Human-Robot Interaction Studies Through Crowdsourcing

Abstract

Human-Robot Interaction (HRI) is a rapidly expanding field of study that focuses on allowing nonroboticist users to naturally and effectively interact with robots. The importance of conducting extensive user studies has become a fundamental component of HRI research; however, due to the nature of robotics research, such studies often become expensive, time consuming, and limited to constrained demographics. In this work, we present the Robot Management System (RMS), a novel framework for bringing robotic experiments to the web. We present a detailed description of our open-source system and describe an initial trial of the RMS as a means of conducting user studies. Using a series of navigation and manipulation tasks with a PR2 robot, we compare three user study conditions: users that are co-present with the robot, users that are recruited to the university laboratory but control the robot from a different room, and remote web-based users. Our findings show little statistical differences between usability patterns across these groups, validating the use of web-based crowdsourcing techniques for certain types of HRI evaluations.

Robots, Pancakes, and Computer Games

“…Other researchers have also been employing simulation environments [25]. While the results in terms of accuracy of the gathered data are promising, these simulators have not been designed to appeal to the general public, to be operated by naïve users, or to support motivation by introducing game mechanics beyond the playful appeal of an interactive virtual world.…”

Towards Integrating Real-Time Crowd Advice with Reinforcement Learning

“…Human control of robots has been previously explored in the context of a robot Ouija board and navigation setting [2]. The Robot Management System [10] (RMS) also uses on-line contributors to crowdsource human-robot interaction studies. RMS used groups of participants working from their home computers to practice controlling a robot using camera views and a webbased control interface.…”

Construction of an object manipulation database from grasp demonstrations

“…The system does not allow for direct control of a robot, and researchers must design tasks to be simple and quick to ensure for high quality data. With new web robotics systems such as the Robot Management System (RMS) [14], we can perform more complicated tasks, including the grasp pose demonstrations used in this paper.…”

“…In this paper, we present the results of grasping data from non-expert users, collected in a crowdsourcing user study. The study consisted of remote users connecting to a PR2 over the Internet, conducted using RobotsFor.Me [14]. We instructed participants to demonstrate as many grasps on as many objects as they could within a 20 minute time limit by teleoperating the robot using a web interface.…”