Learning robots to grasp by demonstration

Coninck, Elias De; Verbelen, Tim; Molle, Pieter Van; Simoens, Pieter; Dhoedt, Bart

doi:10.1016/j.robot.2020.103474

Cited by 26 publications

(16 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [33], the researchers used sequential grasping data to achieve grasp recognition, which shows the potential of history proprioceptive data to realize autonomous grasping. Most of the prior works using human demonstrations focused on the grasp planning that generates the pre-grasp pose and contact points for the target object [10], [32], [34]. While little attention has been paid to the finger control and the grasp execution.…”

Section: Learning From Demonstration For Robotic Graspingmentioning

confidence: 99%

Learning Adaptive Grasping From Human Demonstrations

Wang¹,

Hu²,

Sun³

et al. 2022

IEEE/ASME Trans. Mechatron.

View full text Add to dashboard Cite

This work studied a learning-based approach to learn grasping policies from teleoperated human demonstrations which can achieve adaptive grasping using three different neural network (NN) structures. To transfer human grasping skills effectively, we used multi-sensing state within a sliding time window to learn the state-action mapping. By teleoperating an anthropomorphic robotic hand using human hand tracking, we collected training datasets from representative grasping of various objects, which were used to train grasping policies with three proposed NN structures. The learned policies can grasp objects with varying sizes, shapes, and stiffness. We benchmarked the grasping performance of all policies, and experimental validations showed significant advantages of using the sequential history states, compared to the instantaneous feedback. Based on the benchmark, we further validated the best NN structure to conduct extensive experiments of grasping hundreds of unseen objects with adaptive motions and grasping forces.

show abstract

Section: Learning From Demonstration For Robotic Graspingmentioning

confidence: 99%

Learning Adaptive Grasping From Human Demonstrations

Wang¹,

Hu²,

Sun³

et al. 2022

IEEE/ASME Trans. Mechatron.

View full text Add to dashboard Cite

show abstract

“…Several studies have been proposed to exploit data from captured expert manipulations such as [24], which learns interactions from videos of experts, or [25], [26], which uses custom handheld devices to collect grasping demonstration. Nevertheless, many recent studies [27]- [29] use the Cornell dataset [11] or similarly acquired datasets [30], [31] which provide thousands of grasp locations labeled by humans. These grasp demonstrations have been used to infer an evaluation function that ranks grasp candidates according to the expert specifications [20], [32], [33].…”

Section: Related Workmentioning

confidence: 99%

Data-efficient learning of object-centric grasp preferences

Fleytoux,

Ma,

Ivaldi

et al. 2022

Preprint

View full text Add to dashboard Cite

Grasping made impressive progress during the last few years thanks to deep learning. However, there are many objects for which it is not possible to choose a grasp by only looking at an RGB-D image, might it be for physical reasons (e.g., a hammer with uneven mass distribution) or task constraints (e.g., food that should not be spoiled). In such situations, the preferences of experts need to be taken into account.In this paper, we introduce a data-efficient grasping pipeline (Latent Space GP Selector -LGPS) that learns grasp preferences with only a few labels per object (typically 1 to 4) and generalizes to new views of this object. Our pipeline is based on learning a latent space of grasps with a dataset generated with any state-of-the-art grasp generator (e.g., Dex-Net). This latent space is then used as a low-dimensional input for a Gaussian process classifier that selects the preferred grasp among those proposed by the generator.The results show that our method outperforms both GR-ConvNet and GG-CNN (two state-of-the-art methods that are also based on labeled grasps) on the Cornell dataset, especially when only a few labels are used: only 80 labels are enough to correctly choose 80% of the grasps (885 scenes, 244 objects). Results are similar on our dataset (91 scenes, 28 objects).

show abstract

“…The proposed solution is shown to be capable of generating a viable grasp, on previously unseen objects in 1.3 seconds. De Coninck et al (2020) [7] show the application of a deep learning environment through demonstration, for application in robotic manipulators that cooperate with humans. Authors demonstrate the application of the algorithm on a Franka Panda collaborative robot, with a 90% average success rate of grasp.…”

Section: Introductionmentioning

confidence: 99%

Prediction of Robot Grasp Robustness using Artificial Intelligence Algorithms

et al. 2022

View full text Add to dashboard Cite

Predicting the quality of the robot end-effector grasp quality during an industrial robot manipulator operation can be an extremely complex task. As is often the case with such complex tasks, Artificial Intelligence methods may be applied to attempt the creation of a model -if sufficient data exists. The presented dataset uses a publicly available dataset, consisting of 992632 measurements of position, torque, and velocity -for each of the three joints of three fingers of the simulated end-effector. The dataset is first analyzed and pre-processed to prepare it for model training. The duplicate values are removed from the dataset, as well as the statistical outliers. Then, a multilayer perceptron (MLP) machine learning algorithm is applied to 80% of the data contained in the dataset, using the Grid Search algorithm to determine the best combination of MLP hyperparameters. As the dataset consists of torque, velocity, and speed measurements for separate joints and fingers of the tested end-effector the testing is performed to see if a subset of the inputs may be used to regress the robustness of the given grip. The normalization of the dataset is also applied, and its effect on the regression quality is tested. The results, evaluated with the coefficient of determination, show that while the best model is achieved using all the possible inputs, a satisfactory result can be obtained using only velocity and torque.The results also show that the normalization of the dataset improves the regression quality in all the observed cases.

show abstract

Learning robots to grasp by demonstration

Cited by 26 publications

References 34 publications

Learning Adaptive Grasping From Human Demonstrations

Learning Adaptive Grasping From Human Demonstrations

Data-efficient learning of object-centric grasp preferences

Prediction of Robot Grasp Robustness using Artificial Intelligence Algorithms

Contact Info

Product

Resources

About