H2O: A Benchmark for Visual Human-human Object Handover Analysis

Ye, Ruolin; Xu, Wenqiang; Xue, Zhendong; Tang, Tutian; Wang, Yanfeng; Lu, Cewu

doi:10.1109/iccv48922.2021.01547

Cited by 22 publications

(12 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1). The marker-based datasets collect hand poses with the aid of hand-attached magnetic sensors [15,59,60] or reflective markers [52]. 2).…”

Section: Related Workmentioning

confidence: 99%

“…While the word "affordance" has different formulations in different tasks, in this paper, we denote "affordance" as the functionality of object. Since 2019, there have been at least 9 datasets of hand-object interaction released: ObMan [23], YCBAfford [11], HO3D [19], Con-tactPose [5], GRAB [52], DexYCB [10], two H2O [29,59] and DexMV [42]. However, these datasets lack comprehensive awareness of the object's affordance and the hand's interactions with it.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction

Yang¹,

Li²,

Zhan³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Learning how humans manipulate objects requires machines to acquire knowledge from two perspectives: one for understanding object affordances and the other for learning human's interactions based on the affordances. Even though these two knowledge bases are crucial, we find that current databases lack a comprehensive awareness of them. In this work, we propose a multi-modal and rich-annotated knowledge repository, OakInk, for visual and cognitive understanding of hand-object interactions. We start to collect 1,800 common household objects and annotate their affordances to construct the first knowledge base: Oak. Given the affordance, we record rich human interactions with 100 selected objects in Oak. Finally, we transfer the interactions on the 100 recorded objects to their virtual counterparts through a novel method: Tink. The recorded and transferred hand-object interactions constitute the second knowledge base: Ink. As a result, OakInk contains 50,000 distinct affordance-aware and intent-oriented hand-object interactions. We benchmark OakInk on pose estimation and grasp generation tasks. Moreover, we propose two practical applications of OakInk: intent-based interaction generation and handover generation. Our datasets and source code are publicly available at https://github.com/ lixiny/OakInk.

show abstract

“…1). The marker-based datasets collect hand poses with the aid of hand-attached magnetic sensors [15,59,60] or reflective markers [52]. 2).…”

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction

Yang¹,

Li²,

Zhan³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Our work is also related to recent efforts on standardizing the experimental setting and protocol for handovers. Ye et al [34] proposed a large-scale humanto-human handover dataset with object and hand pose annotations, and used it to study human grasp prediction. Rather than predicting human grasps, Chao et al [15] studied robot grasp generation for safe H2R handovers.…”

Section: Related Workmentioning

confidence: 99%

HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers

Chao¹,

Paxton²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

We introduce a new simulation benchmark "Han-doverSim" for human-to-robot object handovers. To simulate the giver's motion, we leverage a recent motion capture dataset of hand grasping of objects. We create training and evaluation environments for the receiver with standardized protocols and metrics. We analyze the performance of a set of baselines and show a correlation with a real-world evaluation. 1

show abstract

“…Also most givers in human handovers preferred a precision grasp irrespective of object. In [17], an image based data set focused on hand poses and grasps via visual analysis of handovers is described, where the hand poses are tracked by markers placed on each finger.…”

Section: Background and Related Workmentioning

confidence: 99%

Human Inspired Grip-Release Technique for Robot-Human Handovers

Khanna

Björkman

Smith

2022

2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids)

View full text Add to dashboard Cite

Handovers are basic yet sophisticated motor tasks performed seamlessly by humans. They are among the most common activities in our daily lives and social environments. This makes mastering the art of handovers critical for a social and collaborative robot. In this work, we present an experimental study that involved human-human handovers by 13 pairs, i.e., 26 participants. We record and explore multiple features of handovers amongst humans aimed at inspiring handovers amongst humans and robots. With this work, we further create and publish a novel data set of 8672 handovers, bringing together human motion and the forces involved. We further analyze the effect of object weight and the role of visual sensory input in human-human handovers, as well as possible design implications for robots. As a proof of concept, the data set was used for creating a human-inspired data-driven strategy for robotic grip release in handovers, which was demonstrated to result in better robot to human handovers.

show abstract

H2O: A Benchmark for Visual Human-human Object Handover Analysis

Cited by 22 publications

References 21 publications

OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction

OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction

HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers

Human Inspired Grip-Release Technique for Robot-Human Handovers

Contact Info

Product

Resources

About