Learning Object Grasping for Soft Robot Hands

Choi, Changhyun; Schwarting, Wilko; DelPreto, Joseph; Rus, Daniela

doi:10.1109/lra.2018.2810544

Cited by 141 publications

(119 citation statements)

References 27 publications

Supporting

Mentioning

110

Contrasting

Order By: Relevance

“…Contact is approximated as a single point per fingertip. A large body of work in robotics aims to predict a configuration of the end-effector [32,9,28] suitable for grasping. In contrast to ContactDB, these works model contact as a single point per hand digit, ignoring other contact.…”

Section: Predicting Grasp Contactmentioning

confidence: 99%

ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging

Brahmbhatt

Ham

Kemp

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

147

102

View full text Add to dashboard Cite

Grasping and manipulating objects is an important human skill. Since hand-object contact is fundamental to grasping, capturing it can lead to important insights. However, observing contact through external sensors is challenging because of occlusion and the complexity of the human hand. We present ContactDB, a novel dataset of contact maps for household objects that captures the rich handobject contact that occurs during grasping, enabled by use of a thermal camera. Participants in our study grasped 3D printed objects with a post-grasp functional intent. Con-tactDB includes 3750 3D meshes of 50 household objects textured with contact maps and 375K frames of synchronized RGB-D+thermal images. To the best of our knowledge, this is the first large-scale dataset that records detailed contact maps for human grasps. Analysis of this data shows the influence of functional intent and object size on grasping, the tendency to touch/avoid 'active areas', and the high frequency of palm and proximal finger contact. Finally, we train state-of-the-art image translation and 3D convolution algorithms to predict diverse contact patterns from object shape. Data, code and models are available at https://contactdb.cc.gatech.edu.

show abstract

Section: Predicting Grasp Contactmentioning

confidence: 99%

ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging

Brahmbhatt

Ham

Kemp

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

147

102

View full text Add to dashboard Cite

show abstract

“…Robot pushing [1], grasping [2] or push-grasping [3] has been actively studied but mostly target-agnostic. Without incorporating target information effectively, fast adaptations (e.g., by applying the target mask) of these methods for target-oriented tasks are not successful.…”

Section: Introductionmentioning

confidence: 99%

A Deep Learning Approach to Grasping the Invisible

Yang

Liang

Choi

2020

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

We introduce a new problem named "grasping the invisible", where a robot is tasked to grasp an initially invisible target object via a sequence of non-prehensile (e.g., pushing) and prehensile (e.g., grasping) actions. In this problem, nonprehensile actions are needed to search for the target and rearrange cluttered objects around it. We propose to solve the problem by formulating a deep reinforcement learning approach in an actor-critic format. A critic that maps both the visual observations and the target information to expected rewards of actions is learned via deep Q-learning for instance pushing and grasping. Two actors are proposed to take in the critic predictions and the domain knowledge for two subtasks: a Bayesian-based actor accounting for past experience performs explorational pushing to search for the target; once the target is found, a classifier-based actor coordinates the target-oriented pushing and grasping to grasp the target in clutter. The model is entirely self-supervised through the robot-environment interactions. Our system achieves 93% and 87% task success rate on the two subtasks in simulation and 85% task success rate in real robot experiments, which outperforms several baselines by large margins. Supplementary material is available at: https://sites.google.com/umn.edu/grasping-invisible.

show abstract

“…In the case of a low-DOF gripper, if the neural network predicts the correct direction and orientation towards the object, one can simply close the gripper and the predicted grasp operation will very likely be successful. Therefore, most prior works [23], [37], [7] only learn the approaching direction and orientation of the gripper. For a high-DOF gripper, however, there are multiple remaining DOFs (beyond direction and orientation) to be determined after the wrist pose is known.…”

Section: Introductionmentioning

confidence: 99%

“…However, the high-DOF nature of the gripper results in a large search space for the sampling-based optimizer, making the online phase very computationally costly. In the second kind of method [7], a neural network is trained to predict the grasp poses directly from single-view observations of the object. As a result, this direct method becomes very efficient because only a forward propagation through the neural network is needed to generate the grasp pose.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Generating Grasp Poses for a High-DOF Gripper Using Neural Networks

Liu

Pan

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

We present a learning-based method for representing grasp poses of a high-DOF hand using neural networks. Due to redundancy in such high-DOF grippers, there exists a large number of equally effective grasp poses for a given target object, making it difficult for the neural network to find consistent grasp poses. We resolve this ambiguity by generating an augmented dataset that covers many possible grasps for each target object and train our neural networks using a consistency loss function to identify a one-to-one mapping from objects to grasp poses. We further enhance the quality of neuralnetwork-predicted grasp poses using a collision loss function to avoid penetrations. We use an object dataset that combines the BigBIRD Database, the KIT Database, the YCB Database, and the Grasp Dataset to show that our method can generate high-DOF grasp poses with higher accuracy than supervised learning baselines. The quality of the grasp poses is on par with the groundtruth poses in the dataset. In addition, our method is robust and can handle noisy object models such as those constructed from multi-view depth images, allowing our method to be implemented on a 25-DOF Shadow Hand hardware platform.

show abstract

Learning Object Grasping for Soft Robot Hands

Cited by 141 publications

References 27 publications

ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging

ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging

A Deep Learning Approach to Grasping the Invisible

Generating Grasp Poses for a High-DOF Gripper Using Neural Networks

Contact Info

Product

Resources

About