Human-Robot Mutual Adaptation in Shared Autonomy

Nikolaidis, Stefanos; Zhu, Yuanxin; Hsu, David; Srinivasa, Siddhartha S.

doi:10.1145/2909824.3020252

Cited by 97 publications

(54 citation statements)

References 21 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several existing techniques may accomplish this, including concatenating m previous frames with the current observation, or adding recurrent connections to the copilot policy architecture as in [11]. Finally, users will adapt to the robot's interface, and explicitly capturing this may improve copilot training and inform theoretical guarantees on convergence [23].…”

Section: Discussionmentioning

confidence: 99%

Shared Autonomy via Deep Reinforcement Learning

Reddy

Dragan

Levine

2018

Robotics: Science and Systems XIV

130

View full text Add to dashboard Cite

Abstract-In shared autonomy, user input is combined with semi-autonomous control to achieve a common goal. The goal is often unknown ex-ante, so prior work enables agents to infer the goal from user input and assist with the task. Such methods tend to assume some combination of knowledge of the dynamics of the environment, the user's policy given their goal, and the set of possible goals the user might target, which limits their application to real-world scenarios. We propose a deep reinforcement learning framework for model-free shared autonomy that lifts these assumptions. We use human-in-the-loop reinforcement learning with neural network function approximation to learn an end-to-end mapping from environmental observation and user input to agent action values, with task reward as the only form of supervision. This approach poses the challenge of following user commands closely enough to provide the user with real-time action feedback and thereby ensure high-quality user input, but also deviating from the user's actions when they are suboptimal. We balance these two needs by discarding actions whose values fall below some threshold, then selecting the remaining action closest to the user's input. Controlled studies with users (n = 12) and synthetic pilots playing a video game, and a pilot study with users (n = 4) flying a real quadrotor, demonstrate the ability of our algorithm to assist users with real-time control tasks in which the agent cannot directly access the user's private information through observations, but receives a reward signal and user input that both depend on the user's intent. The agent learns to assist the user without access to this private information, implicitly inferring it from the user's input. This enables the assisted user to complete the task more effectively than the user or an autonomous agent could on their own. This paper is a proof of concept that illustrates the potential for deep reinforcement learning to enable flexible and practical assistive systems.

show abstract

Section: Discussionmentioning

confidence: 99%

Shared Autonomy via Deep Reinforcement Learning

Reddy

Dragan

Levine

2018

Robotics: Science and Systems XIV

130

View full text Add to dashboard Cite

show abstract

“…Trust emerges naturally in collaborative settings. In human-robot collabation [24,23], trust models can be used to enable more natural interactions. For example, Min et al [5] proposed a decision-theoretic model that incorporates a predictive trust model, and showed that policies that took human trust into consideration led to better outcomes.…”

Section: Background and Related Workmentioning

confidence: 99%

The Transfer of Human Trust in Robot Capabilities across Tasks

Soh

Pan

Chen

et al. 2018

Robotics: Science and Systems XIV

Self Cite

View full text Add to dashboard Cite

Abstract-Trust is crucial in shaping human interactions with one another and with robots. This work investigates how human trust in robot capabilities transfers across tasks. We present a human-subjects study of two distinct task domains: a Fetch robot performing household tasks and a virtual reality simulation of an autonomous vehicle performing driving and parking maneuvers. Our findings lead to a functional view of trust and two novel predictive models-a recurrent neural network architecture and a Bayesian Gaussian process-that capture trust evolution and transfer via latent task representations. Experiments show that the two proposed models outperform existing approaches when predicting trust across unseen tasks and participants. These results indicate that (i) a task-dependent functional trust model captures human trust in robot capabilities more accurately, and (ii) trust transfer across tasks can be inferred to a good degree. The latter enables trust-based robot decision-making for fluent human-robot interaction. In particular, our models can be used to derive robot policies that mitigate under-trust or over-trust by human teammates in collaborative multi-task settings.

show abstract

“…Alternatively, robots can also learn about the human while learning from that human. In Nikolaidis et al [10], for instance, the robot learns about the end-user's adaptability in addition to their reward function. Building on these works, we will infer the end-user's teaching strategy, so that the robot can more accurately learn from human interactions.…”

Section: Related Work a Robots Learning From Humansmentioning

confidence: 99%

Enabling Robots to Infer How End-Users Teach and Learn Through Human-Robot Interaction

Losey

O’Malley

2019

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

During human-robot interaction (HRI), we want the robot to understand us, and we want to intuitively understand the robot. In order to communicate with and understand the robot, we can leverage interactions, where the human and robot observe each other's behavior. However, it is not always clear how the human and robot should interpret these actions: a given interaction might mean several different things. Within today's state-of-the-art, the robot assigns a single interaction strategy to the human, and learns from or teaches the human according to this fixed strategy. Instead, we here recognize that different users interact in different ways, and so one size does not fit all. Therefore, we argue that the robot should maintain a distribution over the possible human interaction strategies, and then infer how each individual end-user interacts during the task. We formally define learning and teaching when the robot is uncertain about the human's interaction strategy, and derive solutions to both problems using Bayesian inference. In examples and a benchmark simulation, we show that our personalized approach outperforms standard methods that maintain a fixed interaction strategy.

show abstract

Human-Robot Mutual Adaptation in Shared Autonomy

Cited by 97 publications

References 21 publications

Shared Autonomy via Deep Reinforcement Learning

Shared Autonomy via Deep Reinforcement Learning

The Transfer of Human Trust in Robot Capabilities across Tasks

Enabling Robots to Infer How End-Users Teach and Learn Through Human-Robot Interaction

Contact Info

Product

Resources

About