Correct Me If I am Wrong: Interactive Learning for Robotic Manipulation

Chisari, Eugenio; Welschehold, Tim; Boedecker, Joschka; Burgard, Wolfram; Valada, Abhinav

doi:10.1109/lra.2022.3145516

Cited by 18 publications

(22 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We follow [23] in the setup of our experiments, with the action space being constituted by the change in the robot's end-effector pose. Using an LSTM, we predict the mean of a Gaussian action-distribution with fixed variance, i.e.…”

Section: E Imitation Learningmentioning

confidence: 99%

Self-Supervised Learning of Multi-Object Keypoints for Robotic Manipulation

Hartz¹,

Chisari²,

Welschehold³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In recent years, policy learning methods using either reinforcement or imitation have made significant progress. However, both techniques still suffer from being computationally expensive and requiring large amounts of training data. This problem is especially prevalent in real-world robotic manipulation tasks, where access to ground truth scene features is not available and policies are instead learned from raw camera observations. In this paper, we demonstrate the efficacy of learning image keypoints via the Dense Correspondence pretext task for downstream policy learning. Extending prior work to challenging multi-object scenes, we show that our model can be trained to deal with important problems in representation learning, primarily scale-invariance and occlusion. We evaluate our approach on diverse robot manipulation tasks, compare it to other visual representation learning approaches, and demonstrate its flexibility and effectiveness for sample-efficient policy learning.

show abstract

Section: E Imitation Learningmentioning

confidence: 99%

Self-Supervised Learning of Multi-Object Keypoints for Robotic Manipulation

Hartz¹,

Chisari²,

Welschehold³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…IWR works under the assumption that the teacher is always able to correct bad behaviors, which might not be true in general, since non-expert users might be in charge of training the robot. In Chisari et al (2022) the Corrective and Evaluative Interactive Learning (CEILing) framework combines human interventions with evaluative feedback. The use of evaluative feedback on non-corrected portions of the trajectory gives the human teacher the option to decide which part of the trajectory to use for training and which to discard.…”

Section: Human-gated Interventionsmentioning

confidence: 99%

“…For example, manipulation tasks can be solved by interactively learning the desired end-effector transitions. Those transitions can then be achieved by the robot using a feedback controller, such as cartesian impedance (Franzese et al, 2021b) or velocity (Chisari et al, 2022) controllers.…”

Section: Learning Desired State Transition/dynamicsmentioning

confidence: 99%

“…The desired state transitions can be described as a function of the current state or with a state-independent formulation; for example, as a function of time/phase using movement primitives like Dynamic Movement Primitive (DMP) or Probabilistic Movement Primitive (ProMP) or as a function of the current state space using a Gaussian Process (GP) (Franzese et al, 2021b) or a NN (Chisari et al, 2022) (introduced in Section 6). When learning a state-dependent transition the policy can be described as π s (s t ) = s d t+1 that still fits the MDP formulation.…”

Section: State Dependent Vs State Independent Transitionsmentioning

confidence: 99%

“…Every time the policy is updated, data is sampled from both buffers; hence, the learner improves its behavior from the interventions while avoiding drastic changes in its behavior, as it is also trained from data generated by itself. Chisari et al (2022) extended this idea by including evaluative feedback in the learning framework. Therefore, instead of storing every learner's transition into a buffer, only the transitions with positive feedback are stored, avoiding the policy to learn from its own erroneous behavior.…”

Section: Neural Networkmentioning

confidence: 99%

See 2 more Smart Citations

Interactive Imitation Learning in Robotics: A Survey

Celemin¹,

Pérez-Dattari²,

Chisari³

et al. 2022

Preprint

View full text Add to dashboard Cite

Interactive Imitation Learning (IIL) is a branch of Imitation Learning (IL) where human feedback is provided intermittently during robot execution allowing an online improvement of the robot's behavior.In recent years, IIL has increasingly started to carve out its own space as a promising data-driven alternative for solving complex robotic tasks. The advantages of IIL are twofold, 1) it is data-efficient, as the human feedback guides the robot directly towards an improved behavior (in contrast with Reinforcement Learning (RL), where behaviors must be discovered by trial and error), and 2) it is robust, as the distribution mismatch between the teacher and learner trajectories is minimized by providing feedback directly over the learner's trajectories (as opposed to offline IL methods such as Behavioral Cloning).Nevertheless, despite the opportunities that IIL presents, its terminology, structure, and applicability are not clear nor unified in the literature, slowing down its development and, therefore, the research of innovative formulations and discoveries.In this article, we attempt to facilitate research in IIL and lower entry barriers for new practitioners by providing a survey of the field that unifies and structures it. In addition, we aim to raise awareness of its potential, what has been accomplished and what are still open research questions.

show abstract

Knowledge- and ambiguity-aware robot learning from corrective and evaluative feedback

Celemin

Kober

2023

Neural Comput & Applic

View full text Add to dashboard Cite

In order to deploy robots that could be adapted by non-expert users, interactive imitation learning (IIL) methods must be flexible regarding the interaction preferences of the teacher and avoid assumptions of perfect teachers (oracles), while considering they make mistakes influenced by diverse human factors. In this work, we propose an IIL method that improves the human–robot interaction for non-expert and imperfect teachers in two directions. First, uncertainty estimation is included to endow the agents with a lack of knowledge awareness (epistemic uncertainty) and demonstration ambiguity awareness (aleatoric uncertainty), such that the robot can request human input when it is deemed more necessary. Second, the proposed method enables the teachers to train with the flexibility of using corrective demonstrations, evaluative reinforcements, and implicit positive feedback. The experimental results show an improvement in learning convergence with respect to other learning methods when the agent learns from highly ambiguous teachers. Additionally, in a user study, it was found that the components of the proposed method improve the teaching experience and the data efficiency of the learning process.

show abstract

Correct Me If I am Wrong: Interactive Learning for Robotic Manipulation

Cited by 18 publications

References 60 publications

Self-Supervised Learning of Multi-Object Keypoints for Robotic Manipulation

Self-Supervised Learning of Multi-Object Keypoints for Robotic Manipulation

Interactive Imitation Learning in Robotics: A Survey

Knowledge- and ambiguity-aware robot learning from corrective and evaluative feedback

Contact Info

Product

Resources

About