Shared Autonomy via Deep Reinforcement Learning

Reddy, Siddharth; Dragan, Anca D.; Levine, Sergey

doi:10.48550/arxiv.1802.01744

Cited by 27 publications

(37 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Virtual reality [7,32] can help, but the human effort remains considerable for complex tasks, and when many demonstrations are required. Shared autonomy [9,12,14,25] offers a better solution to collecting large-scale data. These works blend robot and user intent using optimization [9], reinforcement learning [25], and learned coarse-to-fine user precision [14], while ours lets the user look far into the future to understand the autonomous prediction.…”

Section: Related Workmentioning

confidence: 99%

“…Shared autonomy [9,12,14,25] offers a better solution to collecting large-scale data. These works blend robot and user intent using optimization [9], reinforcement learning [25], and learned coarse-to-fine user precision [14], while ours lets the user look far into the future to understand the autonomous prediction. A similar forecasting method was proposed by Liu et al [17], but it is used in a behavior cloning loss function, rather than for communicating intent to the user.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Assistive Tele-op: Leveraging Transformers to Collect Robotic Task Demonstrations

Clever¹,

Handa²,

Mazhar³

et al. 2021

Preprint

View full text Add to dashboard Cite

Sharing autonomy between robots and human operators could facilitate data collection of robotic task demonstrations to continuously improve learned models. Yet, the means to communicate intent and reason about the future are disparate between humans and robots. We present Assistive Tele-op, a virtual reality (VR) system for collecting robot task demonstrations that displays an autonomous trajectory forecast to communicate the robot's intent. As the robot moves, the user can switch between autonomous and manual control when desired. This allows users to collect task demonstrations with both a high success rate and with greater ease than manual teleoperation systems. Our system is powered by transformers, which can provide a window of potential states and actions far into the future -with almost no added computation time. A key insight is that human intent can be injected at any location within the transformer sequence if the user decides that the model-predicted actions are inappropriate. At every time step, the user can (1) do nothing and allow autonomous operation to continue while observing the robot's future plan sequence, or (2) take over and momentarily prescribe a different set of actions to nudge the model back on track. We host the videos and other supplementary material at https://sites.google.com/view/assistive-teleop.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Assistive Tele-op: Leveraging Transformers to Collect Robotic Task Demonstrations

Clever¹,

Handa²,

Mazhar³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In recent years, some model-free RLSC methods have been proposed. The earliest work is [19], where the agent maximizes a combination of task performance and user feedback rewards using Deep-Q learning. In [20], the approach is extended to maximize human-control authority using residual policy learning.…”

Section: B Reinforcement Learning For Shared Controlmentioning

confidence: 99%

Learning to Arbitrate Human and Robot Control using Disagreement between Sub-Policies

Oh¹,

Toussaint²,

Mainprice³

2021

Preprint

View full text Add to dashboard Cite

In the context of teleoperation, arbitration refers to deciding how to blend between human and autonomous robot commands. We present a reinforcement learning solution that learns an optimal arbitration strategy that allocates more control authority to the human when the robot comes across a decision point in the task. A decision point is where the robot encounters multiple options (sub-policies), such as having multiple paths to get around an obstacle or deciding between two candidate goals. By expressing each directional sub-policy as a von Mises distribution, we identify the decision points by observing the modality of the mixture distribution. Our reward function reasons on this modality and prioritizes to match its learned policy to either the user or the robot accordingly. We report teleoperation experiments on reach-and-grasping objects using a robot manipulator arm with different simulated human controllers. Results indicate that our shared control agent outperforms direct control and improves the teleoperation performance among different users. Using our reward term enables flexible blending between human and robot commands while maintaining safe and accurate teleoperation.

show abstract

“…Moreover, the classifiers are trained to predict the labels of all samples in the training set as in full automation. The extensive body of work on human-machine collaboration has predominantly considered settings in which the machine and the human interact with each other [5,16,17,18,19,25,30,34,35,39,42,48,50,51,52,54]. In this context, our work is more closely connected to a line of work that studies switching behavior and switching costs in the context of human-computer interaction [7,20,22,24,26], which we see as complementary.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement Learning Under Algorithmic Triage

Straitouri,

Singla,

Meresht

et al. 2021

Preprint

View full text Add to dashboard Cite

Methods to learn under algorithmic triage have predominantly focused on supervised learning settings where each decision, or prediction, is independent of each other. Under algorithmic triage, a supervised learning model predicts a fraction of the instances and humans predict the remaining ones. In this work, we take a first step towards developing reinforcement learning models that are optimized to operate under algorithmic triage. To this end, we look at the problem through the framework of options and develop a two-stage actor-critic method to learn reinforcement learning models under triage. The first stage performs offline, off-policy training using human data gathered in an environment where the human has operated on their own. The second stage performs on-policy training to account for the impact that switching may have on the human policy, which may be difficult to anticipate from the above human data. Extensive simulation experiments in a synthetic car driving task show that the machine models and the triage policies trained using our two-stage method effectively complement human policies and outperform those provided by several competitive baselines.

show abstract

Shared Autonomy via Deep Reinforcement Learning

Cited by 27 publications

References 0 publications

Assistive Tele-op: Leveraging Transformers to Collect Robotic Task Demonstrations

Assistive Tele-op: Leveraging Transformers to Collect Robotic Task Demonstrations

Learning to Arbitrate Human and Robot Control using Disagreement between Sub-Policies

Reinforcement Learning Under Algorithmic Triage

Contact Info

Product

Resources

About