Human Grasp Classification for Reactive Human-to-Robot Handovers

Yang, Wei; Paxton, Chris; Çakmak, Maya; Fox, Dieter

doi:10.48550/arxiv.2003.06000

Cited by 5 publications

(12 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, our work provides a full pipeline for human-to-robot handovers. Our work shows some similarities to [13]. However, the adopted perception and grasp selection approaches are completely different.…”

Section: Related Workmentioning

confidence: 79%

“…However, the adopted perception and grasp selection approaches are completely different. Yang et al [13] classifies human grasp poses and adapts the robot's trajectory accordingly, i.e., their neural network can detect seven different grasp poses and grab a held cube at its centre. Alternatively, our approach implements an object-independent grasp planning algorithm that selects the best picking location based on the object's shape and therefore allows the transfer of unknown objects.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Object-Independent Human-to-Robot Handovers using Real Time Robotic Vision

Rosenberger¹,

Cosgun²,

Newbury³

et al. 2020

Preprint

View full text Add to dashboard Cite

We present an approach for safe and objectindependent human-to-robot handovers using real time robotic vision and manipulation. We aim for general applicability with a generic object detector, a fast grasp selection algorithm and by using a single gripper-mounted RGB-D camera, hence not relying on external sensors. The robot is controlled via visual servoing towards the object of interest. Putting a high emphasis on safety, we use two perception modules: human body part segmentation and hand/finger segmentation. Pixels that are deemed to belong to the human are filtered out from candidate grasp poses, hence ensuring that the robot safely picks the object without colliding with the human partner. The grasp selection and perception modules run concurrently in real-time, which allows monitoring of the progress. In experiments with 13 objects, the robot was able to successfully take the object from the human in 81.9% of the trials.

show abstract

Section: Related Workmentioning

confidence: 79%

Section: Related Workmentioning

confidence: 99%

Object-Independent Human-to-Robot Handovers using Real Time Robotic Vision

Rosenberger¹,

Cosgun²,

Newbury³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…For the third-person perspective, the visual system which analyzes the human-human handover process can be used for demonstration of robot-related tasks, such as human-robot handover [32,26], robot-human handover [4,21,27], dexterous manipulation [22,23], etc. Besides, it can also serve as a analysis tool for cognitive study on human behaviour understanding [20,8].…”

Section: Perception In Handover Taskmentioning

confidence: 99%

“…Being aware of the markerless preference for some research topics [15,3], we also provide a photorealistic H2O-Syn dataset, where the poses of hand and object are transferred from H2O, and the textures of hand and object are scanned from the real counterparts. As human beings are generally the experts that an intelligent robot should learn the skills from, we believe this human-human interaction dataset can also serve as the demonstrations of various manipulation tasks for robots, such as human-robot handover [32,26], robot-human handover [4,21,27], and dexterous manipulation [22,23].…”

Section: Introductionmentioning

confidence: 99%

H2O: A Benchmark for Visual Human-human Object Handover Analysis

Ye¹,

Xu²,

Xue³

et al. 2021

Preprint

View full text Add to dashboard Cite

Object handover is a common human collaboration behavior that attracts attention from researchers in Robotics and Cognitive Science. Though visual perception plays an important role in the object handover task, the whole handover process has been specifically explored. In this work, we propose a novel rich-annotated dataset, H2O, for visual analysis of human-human object handovers. The H2O, which contains 18K video clips involving 15 people who hand over 30 objects to each other, is a multi-purpose benchmark. It can support several vision-based tasks, from which, we specifically provide a baseline method, RGPNet, for a less-explored task named Receiver Grasp Prediction. Extensive experiments show that the RGPNet can produce plausible grasps based on the giver's hand-object states in the pre-handover phase. Besides, we also report the hand and object pose errors with existing baselines and show that the dataset can serve as the video demonstrations for robot imitation learning on the handover task. Dataset, model and code will be made public.

show abstract

“…Recent advances in deep learning-based machine perception have substantially improved the accuracies of pose estimation from real-world sensory data [2][3][4][5][6]. The estimated 6-DoF object pose, represented by the translation and rotation in SE(3), serves as a compact and informative state representation for a variety of downstream tasks, such as robot grasping and manipulation [7], human-robot interactions [8], online camera calibration [9], and tele-presence robot control [10].…”

Section: Introductionmentioning

confidence: 99%

Fast Uncertainty Quantification for Deep Object Pose Estimation

Shi¹,

Zhu²,

Tremblay³

et al. 2020

Preprint

View full text Add to dashboard Cite

Deep learning-based object pose estimators are often unreliable and overconfident especially when the input image is outside the training domain, for instance, with sim2real transfer. Efficient and robust uncertainty quantification (UQ) in pose estimators is critically needed in many robotic tasks. In this work, we propose a simple, efficient, and plug-and-play UQ method for 6-DoF object pose estimation. We ensemble 2-3 pre-trained models with different neural network architectures and/or training data sources, and compute their average pairwise disagreement against one another to obtain the uncertainty quantification. We propose four disagreement metrics, including a learned metric, and show that the average distance (ADD) is the best learning-free metric and it is only slightly worse than the learned metric, which requires labeled target data. Our method has several advantages compared to the prior art: 1) our method does not require any modification of the training process or the model inputs; and 2) it needs only one forward pass for each model. We evaluate the proposed UQ method on three tasks where our uncertainty quantification yields much stronger correlations with pose estimation errors than the baselines. Moreover, in a real robot grasping task, our method increases the grasping success rate from 35% to 90%. Video and code are available at https://sites.google.com/view/fastuq.

show abstract

Human Grasp Classification for Reactive Human-to-Robot Handovers

Cited by 5 publications

References 57 publications

Object-Independent Human-to-Robot Handovers using Real Time Robotic Vision

Object-Independent Human-to-Robot Handovers using Real Time Robotic Vision

H2O: A Benchmark for Visual Human-human Object Handover Analysis

Fast Uncertainty Quantification for Deep Object Pose Estimation

Contact Info

Product

Resources

About