DexTransfer: Real World Multi-fingered Dexterous Grasping with Minimal Human Demonstrations

Chen, Zoey Qiuyu; Wyk, Karl Van; Chao, Yu-Wei; Yang, Wei; Mousavian, Arsalan; Gupta, Abhishek; Fox, Dieter

doi:10.48550/arxiv.2209.14284

Cited by 1 publication

(1 citation statement)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We find that by including the arm in the optimization process, the position error caused by the structural difference between the human hand and the robotic hand can be reduced. After optimization, we convert the optimized joint angles to actions in MuJoCo and perform correlated sampling [24] on actions in case of dropping the object when lifting without sufficient grasping force. Finally, we execute the refined action sequences in simulation in order to collect demonstrations D composed of state-action pairs (s, a), which are for behavior cloning in the next stage.…”

Section: B Learning Dexrepnetmentioning

confidence: 99%

DexRepNet: Learning Dexterous Robotic Grasping Network with Geometric and Spatial Hand-Object Representations

Liu¹,

Chen²,

Sun³

et al. 2023

Preprint

View full text Add to dashboard Cite

Robotic dexterous grasping is a challenging problem due to the high degree of freedom (DoF) and complex contacts of multi-fingered robotic hands. Existing deep reinforcement learning (DRL) based methods leverage human demonstrations to reduce sample complexity due to the high dimensional action space with dexterous grasping. However, less attention has been paid to hand-object interaction representations for high-level generalization. In this paper, we propose a novel geometric and spatial hand-object interaction representation, named DexRep, to capture dynamic object shape features and the spatial relations between hands and objects during grasping. DexRep comprises Occupancy Feature for rough shapes within sensing range by moving hands, Surface Feature for changing hand-object surface distances, and Local-Geo Feature for local geometric surface features most related to potential contacts. Based on the new representation, we propose a dexterous deep reinforcement learning method to learn a generalizable grasping policy DexRepNet. Experimental results show that our method outperforms baselines using existing representations for robotic grasping dramatically both in grasp success rate and convergence speed. It achieves a 93% grasping success rate on seen objects and higher than 80% grasping success rates on diverse objects of unseen categories in both simulation and real-world experiments.

show abstract

Section: B Learning Dexrepnetmentioning

confidence: 99%