Generative Adversarial Imitation Learning with Deep P-Network for Robotic Cloth Manipulation

Tsurumine, Yoshihisa; Cui, Yunduan; Yamazaki, Kimitoshi; Matsubara, Takamitsu

doi:10.1109/humanoids43949.2019.9034991

Cited by 16 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Building on top of Deep Deterministic Policy Gradients and Hindsight Experience Replay, Nair et al [272] proposed behavior cloning Loss to increase imitating the demonstrations. Besides Q-learning, Generative Adversarial Imitation Learning [364] proposes P-GAIL that integrates imitation learning into the policy gradient framework. P-GAIL considers both smoothness and causal entropy in policy update by utilizing Deep P-Network [365].…”

Section: Drl Recent Advancesmentioning

confidence: 99%

Deep Reinforcement Learning in Computer Vision: A Comprehensive Survey

Le¹,

Rathour²,

Yamazaki³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep reinforcement learning augments the reinforcement learning framework and utilizes the powerful representation of deep neural networks. Recent works have demonstrated the remarkable successes of deep reinforcement learning in various domains including finance, medicine, healthcare, video games, robotics, and computer vision. In this work, we provide a detailed review of recent and state-of-the-art research advances of deep reinforcement learning in computer vision. We start with comprehending the theories of deep learning, reinforcement learning, and deep reinforcement learning. We then propose a categorization of deep reinforcement learning methodologies and discuss their advantages and limitations. In particular, we divide deep reinforcement learning into seven main categories according to their applications in computer vision, i.e. (i) landmark localization (ii) object detection; (iii) object tracking; (iv) registration on both 2D image and 3D image volumetric data (v) image segmentation; (vi) videos analysis; and (vii) other applications. Each of these categories is further analyzed with reinforcement learning techniques, network design, and performance. Moreover, we provide a comprehensive analysis of the existing publicly available datasets and examine source code availability. Finally, we present some open issues and discuss future research directions on deep reinforcement learning in computer vision.

show abstract

Section: Drl Recent Advancesmentioning

confidence: 99%

Deep Reinforcement Learning in Computer Vision: A Comprehensive Survey

Le¹,

Rathour²,

Yamazaki³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Specifically, during every iteration, online GAIL first minimizes the discrepancy in expected cumulative reward between the expert policy and the learned policy and then maximizes such a discrepancy over a given reward function class in adversary. Online GAIL achieves tremendous empirical success in a variety of fields, such as autonomous driving (Kuefler et al, 2017), human behavior modeling (Merel et al, 2017), natural language processing (Chen et al, 2017), and robotics control (Tsurumine et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

Provably Efficient Generative Adversarial Imitation Learning for Online and Offline Setting with Linear Function Approximation

Liu,

Zhang,

et al. 2021

Preprint

View full text Add to dashboard Cite

In generative adversarial imitation learning (GAIL), the agent aims to learn a policy from an expert demonstration so that its performance cannot be discriminated from the expert policy on a certain predefined reward set. In this paper, we study GAIL in both online and offline settings with linear function approximation, where both the transition and reward function are linear in the feature maps. Besides the expert demonstration, in the online setting the agent can interact with the environment, while in the offline setting the agent only accesses an additional dataset collected by a prior. For online GAIL, we propose an optimistic generative adversarial policy optimization algorithm (OGAP) and prove that OGAP achieves O(H 2 d 3/2 K 1/2 +KH 3/2 dN −1/2 1 ) regret. Here N1 represents the number of trajectories of the expert demonstration, d is the feature dimension, and K is the number of episodes. For offline GAIL, we propose a pessimistic generative adversarial policy optimization algorithm (PGAP). For an arbitrary additional dataset, we obtain the optimality gap of PGAP, achieving the minimax lower bound in the utilization of the additional dataset. Assuming sufficient coverage on the additional dataset, we show that PGAP achieves O(H) optimality gap. Here N2 represents the number of trajectories of the additional dataset with sufficient coverage.

show abstract

“…One such task, robotic cloth folding, is of particular interest to robotics community because of its high applicability. After early successes in robotic folding [4], [7], the field is now dominated by deep reinforcement learning approaches [8]- [10] where the most common approach is to perform the training in simulation before transferring the model to a real-world robot via domain randomization [9], [11]- [13]. Challenges for sim-to-real transfer of reinforcement learning policies are well-documented [14].…”

Section: Introductionmentioning

confidence: 99%

“…Sim-to-real is especially difficult for deformable objects because they can not be simulated accurately, and it is difficult to represent their state. Because of these challenges and encouraged by the successful use of expert human demonstrations in reinforcement learning [10], [15], research started exploring how to use demonstration data to train cloth folding policies [5], [11], [16], [17].…”

Section: Introductionmentioning

confidence: 99%

Demonstrating Cloth Folding to Robots: Design and Evaluation of a 2D and a 3D User Interface

Waymouth¹,

Cosgun²,

Newbury³

et al. 2021

Preprint

View full text Add to dashboard Cite

An appropriate user interface to collect human demonstration data for deformable object manipulation has been mostly overlooked in the literature. We present an interaction design for demonstrating cloth folding to robots. Users choose pick and place points on the cloth and can preview a visualization of a simulated cloth before real-robot execution. Two interfaces are proposed: A 2D display-and-mouse interface where points are placed by clicking on an image of the cloth, and a 3D Augmented Reality interface where the chosen points are placed by hand gestures. We conduct a user study with 18 participants, in which each user completed two sequential folds to achieve a cloth goal shape. Results show that while both interfaces were acceptable, the 3D interface was found to be more suitable for understanding the task, and the 2D interface suitable for repetition. Results also found that fold previews improve three key metrics: task efficiency, the ability to predict the final shape of the cloth and overall user satisfaction.

show abstract

Generative Adversarial Imitation Learning with Deep P-Network for Robotic Cloth Manipulation

Cited by 16 publications

References 14 publications

Deep Reinforcement Learning in Computer Vision: A Comprehensive Survey

Deep Reinforcement Learning in Computer Vision: A Comprehensive Survey

Provably Efficient Generative Adversarial Imitation Learning for Online and Offline Setting with Linear Function Approximation

Demonstrating Cloth Folding to Robots: Design and Evaluation of a 2D and a 3D User Interface

Contact Info

Product

Resources

About