Deep Learning for Detecting Robotic Grasps

Lenz, Ian; Lee, Honglak; Saxena, Ashutosh

doi:10.48550/arxiv.1301.3592

Cited by 8 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, [20] uses vision based features (edge and texture filter responses) and learns a logistic regressor over synthetic data. On the other hand, [22], [23] use human annotated grasp data to train grasp synthesis models over RGB-D data. However, as discussed above, large-scale collection of training data for the task of grasp prediction is not trivial and has several issues.…”

Section: Related Workmentioning

confidence: 99%

Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours

Pinto

Gupta

2016

2016 IEEE International Conference on Robotics and Automation (ICRA)

940

823

View full text Add to dashboard Cite

Abstract-Current learning-based robot grasping approaches exploit human-labeled datasets for training the models. However, there are two problems with such a methodology: (a) since each object can be grasped in multiple ways, manually labeling grasp locations is not a trivial task; (b) human labeling is biased by semantics. While there have been attempts to train robots using trial-and-error experiments, the amount of data used in such experiments remains substantially low and hence makes the learner prone to over-fitting. In this paper, we take the leap of increasing the available training data to 40 times more than prior work, leading to a dataset size of 50K data points collected over 700 hours of robot grasping attempts. This allows us to train a Convolutional Neural Network (CNN) for the task of predicting grasp locations without severe overfitting. In our formulation, we recast the regression problem to an 18-way binary classification over image patches. We also present a multi-stage learning approach where a CNN trained in one stage is used to collect hard negatives in subsequent stages. Our experiments clearly show the benefit of using large-scale datasets (and multi-stage training) for the task of grasping. We also compare to several baselines and show state-of-the-art performance on generalization to unseen objects for grasping.

show abstract

Section: Related Workmentioning

confidence: 99%

Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours

Pinto

Gupta

2016

2016 IEEE International Conference on Robotics and Automation (ICRA)

940

823

View full text Add to dashboard Cite

show abstract

“…While some models directly estimate 6dof gripper poses from 3D inputs such as pointclouds, others estimate 2D gripper poses from depth or RGB images and project them to 3D space. The availability of standardised grasp datasets such as Cornell [60] and Jacquard [61] and its relative speed of detection has made the 2D input models a popular choice for application in robotic grasping. These 2D input models can also be categorised based on the type of outputs produced.…”

Section: Grasp Estimation With Convolutional Neural Networkmentioning

confidence: 99%

Robot-Assisted Glovebox Teleoperation for Nuclear Industry

et al. 2021

View full text Add to dashboard Cite

The nuclear industry has some of the most extreme environments in the world, with radiation levels and extremely harsh conditions restraining human access to many facilities. One method for enabling minimal human exposure to hazards under these conditions is through the use of gloveboxes that are sealed volumes with controlled access for performing handling. While gloveboxes allow operators to perform complex handling tasks, they put operators at considerable risk from breaking the confinement and, historically, serious examples including punctured gloves leading to lifetime doses have occurred. To date, robotic systems have had relatively little impact on the industry, even though it is clear that they offer major opportunities for improving productivity and significantly reducing risks to human health. This work presents the challenges of robotic and AI solutions for nuclear gloveboxes, and introduces a step forward for bringing cutting-edge technology to gloveboxes. The problem statement and challenges are highlighted and then an integrated demonstrator is proposed for robotic handling in nuclear gloveboxes for nuclear material handling. The proposed approach spans from tele-manipulation to shared autonomy, computer vision solutions for robotic manipulation to machine learning solutions for condition monitoring.

show abstract

“…A comprehensive literature review of this area can be found in [17,18]. Recently, data-driven learning-based approaches have started to appear, with initial work focused on using human annotators [19]. However, in this work we are more interested in building a self-supervision system [20,21,22,23,24].…”

Section: Robotic Tasksmentioning

confidence: 99%

The Curious Robot: Learning Visual Representations via Physical Interactions

Pinto

Gandhi

Han

et al. 2016

Lecture Notes in Computer Science

121

122

View full text Add to dashboard Cite

Abstract. What is the right supervisory signal to train visual representations? Current approaches in computer vision use category labels from datasets such as ImageNet to train ConvNets. However, in case of biological agents, visual representation learning does not require millions of semantic labels. We argue that biological agents use physical interactions with the world to learn visual representations unlike current vision systems which just use passive observations (images and videos downloaded from web). For example, babies push objects, poke them, put them in their mouth and throw them to learn representations. Towards this goal, we build one of the first systems on a Baxter platform that pushes, pokes, grasps and observes objects in a tabletop environment. It uses four different types of physical interactions to collect more than 130K datapoints, with each datapoint providing supervision to a shared ConvNet architecture allowing us to learn visual representations. We show the quality of learned representations by observing neuron activations and performing nearest neighbor retrieval on this learned representation. Quantitatively, we evaluate our learned ConvNet on image classification tasks and show improvements compared to learning without external data. Finally, on the task of instance retrieval, our network outperforms the ImageNet network on recall@1 by 3%.

show abstract

Deep Learning for Detecting Robotic Grasps

Cited by 8 publications

References 0 publications

Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours

Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours

Robot-Assisted Glovebox Teleoperation for Nuclear Industry

The Curious Robot: Learning Visual Representations via Physical Interactions

Contact Info

Product

Resources

About