Shared autonomy via hindsight optimization for teleoperation and teaming

Abstract

In shared autonomy, a user and autonomous system work together to achieve shared goals. To collaborate effectively, the autonomous system must know the user's goal. As such, most prior works follow a predict-then-act model, first predicting the user's goal with high confidence, then assisting given that goal. Unfortunately, confidently predicting the user's goal may not be possible until they have nearly achieved it, causing predict-then-act methods to provide little assistance. However, the system can often provide useful assistance even when confidence for any single goal is low (e.g. move towards multiple goals). In this work, we formalize this insight by modelling shared autonomy as a Partially Observable Markov Decision Process (POMDP), providing assistance that minimizes the expected cost-to-go with an unknown goal. As solving this POMDP optimally is intractable, we use hindsight optimization to approximate. We apply our framework to both shared-control teleoperation and human-robot teaming. Compared to predict-then-act methods, our method achieves goals faster, requires less user input, decreases user idling time, and results in fewer user-robot collisions.Prepared using sagej.cls [Version: 2015/06/09 v1.01] arXiv:1706.00155v1 [cs.RO] 1 Jun 2017 * While we assume the goal is fixed, we do not assume how the user will achieve that goal (e.g. grasp location) is fixed.

Learning Models of Sequential Decision-Making with Partial Specification of Agent Behavior

“…As mentioned in the introduction, a key motivation behind learning agent models is to enable effective human-machine interaction. The algorithms presented in this work can be used both by a collaborative robot (for learning models of human behavior to improve robot decision-making) and by a human (to learn transparent models of robot behavior to better calibrate trust) (Javdani et al 2018;Yang et al 2017). Both of these use cases offer novel research avenues, such as the efficient specification of decision-theoretic models for interaction (such as, POMDPs and decentralized-POMDPs) and evaluation of the utility of aligned models as compared to purely predictive models for human-machine interaction (Oliehoek, Amato, and others 2016).…”

Probabilistic Human Intent Recognition for Shared Autonomy in Assistive Robotics

“…Memory-based inference is utilized in previous works involving shared autonomy and human-robot systems [8,9,30]. Another approach uses Laplace's approximation [9] and formulates the problem as one of optimizing a partially observable Markov decision process (POMDP) over the user's goal to arbitrate control over a distribution of possible outcomes [20]. The approach considers user inputs for the prediction model and uses a hand-specified distance-based user cost function to achieve a closed-form value function computation.…”

“…In addition to a distancebased likelihood observation used in the prior works, our approach incorporates a fusion of multiple observations and introduces a probabilistic modeling of user control inputs as goal-directed actions that customizes the rationality index value to each individual user, and thus can account for their particular behavior. The motivation for our approach are prior studies that show that users vary greatly in their performance, preferences, and desires [9,20,37]-suggesting a need for assistive systems to customize to individual users.…”

Active Intent Disambiguation for Shared Control Robots

“…Intent can also be inferred from the user's control signals and other environmental cues using various algorithms [11]. Within the context of shared autonomy a Bayesian scheme for user intent prediction models the user within the Markov Decision Process framework [12], [13], [14] and is typically assumed to be noisily optimizing some cost function for their intended goal. In low-dimensional spaces, this cost function can be learned from expert demonstrations using Inverse Reinforcement Learning [15].…”