A Skeleton-Driven Neural Occupancy Representation for Articulated Hands

Karunratanakul, Korrawe; Spurr, Adrian; Fan, Zicong; Hilliges, Otmar; Tang, Siyu

doi:10.1109/3dv53792.2021.00012

Cited by 35 publications

(12 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A key problem in computer vision is to understand how humans interact with their surroundings. Because hands are our primary means of manipulation with the physical world, there has been an intense interest in hand-object pose estimation [5, 13-15, 19, 38, 39] and the synthesis of static grasps for a given object [19,21,24,38]. However, human grasping is not limited to a single time instance, but involves a continuous interaction with objects in order to move them.…”

Section: Introductionmentioning

confidence: 99%

D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions

Christen

Kocabas

Aksan

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

We introduce the dynamic grasp synthesis task: given an object with a known 6D pose and a grasp reference, our goal is to generate motions that move the object to a target 6D pose. This is challenging, because it requires reasoning about the complex articulation of the human hand and the intricate physical interaction with the object. We propose a novel method that frames this problem in the reinforcement learning framework and leverages a physics simulation, both to learn and to evaluate such dynamic interactions. A hierarchical approach decomposes the task into low-level grasping and high-level motion synthesis. It can be used to generate novel hand sequences that approach, grasp, and move an object to a desired location, while retaining human-likeness. We show that our approach leads to stable grasps and generates a wide range of motions. Furthermore, even imperfect labels can be corrected by our method to generate dynamic interaction sequences. Video and code are available at: https://eth-ait.github.io/d-grasp/.

show abstract

Section: Introductionmentioning

confidence: 99%

D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions

Christen

Kocabas

Aksan

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

show abstract

“…We use the trimesh library to detect whether there exists a collision between the hand mesh and the object mesh and compute the max penetration depth between two meshes. We follow the same process as [25,26] to compute I v .…”

Section: Discussionmentioning

confidence: 99%

“…For more accurate reconstruction of hands and manipulated objects, we attempt to combine the advantages of the parametric models and SDFs. Along this direction, previous works [10,14,25,56] attempt to leverage parametric models to learn SDFs from 3D poses or raw scans. In our work, we address a different and more challenging setup of reconstructing hands and objects from monocular RGB images.…”

Section: Introductionmentioning

confidence: 99%

AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstruction

Chen¹,

Hasson²,

Schmid³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent work achieved impressive progress towards joint reconstruction of hands and manipulated objects from monocular color images. Existing methods focus on two alternative representations in terms of either parametric meshes or signed distance fields (SDFs). On one side, parametric models can benefit from prior knowledge at the cost of limited shape deformations and mesh resolutions. Mesh models, hence, may fail to precisely reconstruct details such as contact surfaces of hands and objects. SDF-based methods, on the other side, can represent arbitrary details but are lacking explicit priors. In this work we aim to improve SDF models using priors provided by parametric representations. In particular, we propose a joint learning framework that disentangles the pose and the shape. We obtain hand and object poses from parametric models and use them to align SDFs in 3D space. We show that such aligned SDFs better focus on reconstructing shape details and improve reconstruction accuracy both for hands and objects. We evaluate our method and demonstrate significant improvements over the state of the art on the challenging ObMan and DexYCB benchmarks.

show abstract

“…More directly related are methods that attempt to generate static grasps given an object and sometimes also information about the hand [1, 2, 20-22, 39, 46]. Generally, these approaches either predict a contact map on the object [1,2,20] or synthesize the joint-angle configuration of the grasping hand [21,22,39,46]. [20] propose a hybrid method, where predicted contact maps on objects are used to refine an initial grasp prediction.…”

Section: Related Workmentioning

confidence: 99%

“…A key problem in computer vision is to understand how humans interact with their surroundings. Because hands are our primary means of manipulation with the physical world, there has been an intense interest in hand-object pose estimation [5, 14-16, 19, 39, 40] and the synthesis of static grasps for a given object [19,21,25,39]. However, human grasping is not limited to a single time instance, but involves a continuous interaction with objects in order to move them.…”

Section: Introductionmentioning

confidence: 99%

D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions

Christen¹,

Kocabas²,

Aksan³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We introduce the dynamic grasp synthesis task: given an object with a known 6D pose and a grasp reference, our goal is to generate motions that move the object to a target 6D pose. This is challenging, because it requires reasoning about the complex articulation of the human hand and the intricate physical interaction with the object. We propose a novel method that frames this problem in the reinforcement learning framework and leverages a physics simulation, both to learn and to evaluate such dynamic interactions. A hierarchical approach decomposes the task into low-level grasping and high-level motion synthesis. It can be used to generate novel hand sequences that approach, grasp, and move an object to a desired location, while retaining human-likeness. We show that our approach leads to stable grasps and generates a wide range of motions. Furthermore, even imperfect labels can be corrected by our method to generate dynamic interaction sequences. Video is available at https://eth-ait.github.io/d-grasp/.

show abstract

A Skeleton-Driven Neural Occupancy Representation for Articulated Hands

Cited by 35 publications

References 62 publications

D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions

D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions

AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstruction

D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions

Contact Info

Product

Resources

About