Shared Autonomy via Hindsight Optimization

Abstract

Abstract-In shared autonomy, user input and robot autonomy are combined to control a robot to achieve a goal. Often, the robot does not know a priori which goal the user wants to achieve, and must both predict the user's intended goal, and assist in achieving that goal. We formulate the problem of shared autonomy as a Partially Observable Markov Decision Process with uncertainty over the user's goal. We utilize maximum entropy inverse optimal control to estimate a distribution over the user's goal based on the history of inputs. Ideally, the robot assists the user by solving for an action which minimizes the expected cost-to-go for the (unknown) goal. As solving the POMDP to select the optimal action is intractable, we use hindsight optimization to approximate the solution. In a user study, we compare our method to a standard predict-then-blend approach. We find that our method enables users to accomplish tasks more quickly while utilizing less input. However, when asked to rate each system, users were mixed in their assessment, citing a tradeoff between maintaining control authority and accomplishing tasks quickly.

Learning Reward Functions by Integrating Human Demonstrations and Preferences

“…IRL assumes access to high-quality demonstrations of the task. However, this is rarely available in robotics, where it is difficult to control high degree-of-freedom (DOF) robots [15,27,29]. Preference-based learning methods on the other are hand are very inefficient since they attempt to learn a continuous reward function from binary feedback.…”

Quantifying Hypothesis Space Misspecification in Learning From Human–Robot Demonstrations and Physical Corrections

“…In such interaction paradigms, the robot aims to infer a cost function or policy that best describes the examples that it has received. New avenues of research focus on learning such robot objectives from human input through demonstrations [9], [10], teleoperation data [11], corrections [12], [13], comparisons [14], examples of what constitutes a goal [15], or even specified proxy objectives [16]. In this paper, we focus on learning from two of such types of human inputdemonstrations and physical corrections -although we stress that the principles outlined in our formalism are more general and could be applied to the other interaction modes mentioned.…”

“…To approximate the intractable integral in (12), we sampled a set X of 1500 trajectories. We sampled costs according to (11) given by random unit norm θs, then optimized them with an off-the-shelf trajectory optimizer. We used TrajOpt [41], which is based on sequential quadratic programming and uses convex-convex collision checking.…”

“…The planning and inference were done for robot trajectories in 7-dimensional configuration space, accounting for all relevant constraints including joint limits and self-collisions, as well as collisions between obstacles in the workspace and any part of the robots body. 11 A. Experiment design 1) Independent variables: We used a 2 by 2 factoral design.…”

“…In Tasks 3 and 4, the robot also took a cup from the human and placed it on the table, but this time it initially grasped the cup at the wrong angle, requiring human assistance to correct end-effector orientation to an upright position. For Tasks 1 and 3, the robot knew 11 For video footage of the experiment, see: https://youtu.be/stnFye8HdcU about the feature the human was asked to correct for (E = 1) and participants were told that the robot should be compliant. For Tasks 2 and 4, the correction was poorly explained (E = 0) and participants were instructed to correct any additional unwanted changes in the trajectory.…”