Learning from Interventions: Human-robot interaction as both explicit and implicit feedback

Spencer, Jonathan; Choudhury, Sanjiban; Barnes, Matt; Schmittle, Matthew; Chiang, Mung; Ramadge, Peter J.; Srinivasa, Siddhartha S.

doi:10.15607/rss.2020.xvi.055

Cited by 27 publications

(28 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The approach presented in this paper focuses on when the corrections are generalized beyond kinematic variables and inputted using an external controller. Existing methods for corrections using an external controller [11], [12], [13] allow for path transformations and policy learning of UAVs and mobile robots. However, similar to the methods in pHRI, the main emphasis is on corrections to kinematics and on updating the trajectory or policy rather than using task knowledge to inform possible corrective input.…”

Section: Related Workmentioning

confidence: 99%

Informing Real-Time Corrections in Corrective Shared Autonomy Through Expert Demonstrations

Hagenow

Senft

Radwin

et al. 2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Corrective Shared Autonomy is a method where human corrections are layered on top of an otherwise autonomous robot behavior. Specifically, a Corrective Shared Autonomy system leverages an external controller to allow corrections across a range of task variables (e.g., spinning speed of a tool, applied force, path) to address the specific needs of a task. However, this inherent flexibility makes the choice of what corrections to allow at any given instant difficult to determine. This choice of corrections includes determining appropriate robot state variables, scaling for these variables, and a way to allow a user to specify the corrections in an intuitive manner. This paper enables efficient Corrective Shared Autonomy by providing an automated solution based on Learning from Demonstration to both extract the nominal behavior and address these core problems. Our evaluation shows that this solution enables users to successfully complete a surface cleaning task, identifies different strategies users employed in applying corrections, and points to future improvements for our solution.

show abstract

Section: Related Workmentioning

confidence: 99%

Informing Real-Time Corrections in Corrective Shared Autonomy Through Expert Demonstrations

Hagenow

Senft

Radwin

et al. 2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…There has also been significant recent interest in active preference queries for learning reward functions from pairwise preferences over demonstrations [7,10,13,19,32,39]. However, many forms of human advice can be unintuitive, since the learner may visit states that are significantly far from those the human supervisor would visit, making it difficult for humans to judge what correct behavior looks like without interacting with the environment themselves [36,42].…”

Section: Background and Related Workmentioning

confidence: 99%

“…Kurenkov et al [26] and Xie et al [48] leverage interventions from suboptimal supervisors to accelerate policy learning, but assume that the supervisors are algorithmic and thus can be queried cheaply. Amir et al [2], Kahn et al [23], Kelly et al [24], Spencer et al [42], and Wang et al [47] instead consider learning from human supervisors and present learning algorithms which utilize the timing and nature of human interventions to update the learned policy. By giving the human control for multiple timesteps in a row, these algorithms show improvements over methods that only hand over control on a state-by-state basis [6].…”

Section: Background and Related Workmentioning

confidence: 99%

“…Imitation learning allows a robot to learn from human feedback and examples [3,4,31]. In particular, interactive imitation learning (IL) [24,42,49], in which a human supervisor periodically takes control of the robotic system during policy learning, has emerged as a popular imitation learning method, as interventions are a particularly intuitive form of human feedback [42]. However, a key challenge in interactive imitation learning is to reduce the burden that interventions place on the human supervisor [24,49].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

LazyDAgger: Reducing Context Switching in Interactive Imitation Learning

Hoque

Balakrishna

Putterman

et al. 2021

Preprint

View full text Add to dashboard Cite

Corrective interventions while a robot is learning to automate a task provide an intuitive method for a human supervisor to assist the robot and convey information about desired behavior. However, these interventions can impose significant burden on a human supervisor, as each intervention interrupts other work the human is doing, incurs latency with each context switch between supervisor and autonomous control, and requires time to perform. We present LazyDAgger, which extends the interactive imitation learning (IL) algorithm SafeDAgger to reduce context switches between supervisor and autonomous control. We find that LazyDAgger improves the performance and robustness of the learned policy during both learning and execution while limiting burden on the supervisor. Simulation experiments suggest that LazyDAgger can reduce context switches by an average of 60% over SafeDAgger on 3 continuous control tasks while maintaining state-of-the-art policy performance. In physical fabric manipulation experiments with an ABB YuMi robot, LazyDAgger reduces context switches by 60% while achieving a 60% higher success rate than SafeDAgger at execution time.

show abstract

“…However, there is an inherent cost of time and effort to querying the user for input and so the user-in-the-loop learning process can be prohibitively frustrating [17], [18]. The user tends to have a preference towards online learning approaches (e.g., [14]) that do not require post-hoc corrections to the learned robot policies. Therefore, in this work, we improve task learning during run-time by soliciting different types of demonstrations from the human teacher.…”

Section: Related Workmentioning

confidence: 99%

Ergodic imitation: Learning from what to do and what not to do

Kalinowska

Prabhakar

Fitzsimons

et al. 2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

With growing access to versatile robotics, it is beneficial for end users to be able to teach robots tasks without needing to code a control policy. One possibility is to teach the robot through successful task executions. However, nearoptimal demonstrations of a task can be difficult to provide and even successful demonstrations can fail to capture task aspects key to robust skill replication. Here, we propose a learning from demonstration (LfD) approach that enables learning of robust task definitions without the need for near-optimal demonstrations. We present a novel algorithmic framework for learning tasks based on the ergodic metric-a measure of information content in motion. Moreover, we make use of negative demonstrations-demonstrations of what not to do-and show that they can help compensate for imperfect demonstrations, reduce the number of demonstrations needed, and highlight crucial task elements improving robot performance. In a proofof-concept example of cart-pole inversion, we show that negative demonstrations alone can be sufficient to successfully learn and recreate a skill. Through a human subject study with 24 participants, we show that consistently more information about a task can be captured from combined positive and negative (posneg) demonstrations than from the same amount of just positive demonstrations. Finally, we demonstrate our learning approach on simulated tasks of target reaching and table cleaning with a 7-DoF Franka arm. Our results point towards a future with robust, data-efficient LfD for novice users.

show abstract

Learning from Interventions: Human-robot interaction as both explicit and implicit feedback

Cited by 27 publications

References 19 publications

Informing Real-Time Corrections in Corrective Shared Autonomy Through Expert Demonstrations

Informing Real-Time Corrections in Corrective Shared Autonomy Through Expert Demonstrations

LazyDAgger: Reducing Context Switching in Interactive Imitation Learning

Ergodic imitation: Learning from what to do and what not to do

Contact Info

Product

Resources

About