Leveraging Photometric Consistency Over Time for Sparsely Supervised Hand-Object Reconstruction

Grasping Field: Learning Implicit Representations for Human Grasps

“…Reconstructing hand and object jointly has been studied with both RGB input and RGB-D input [62,71,72,73,78,83,86,87,88,89]. Recently, Hasson et al [27,29] achieved promising results on explicitly modeling the contact by combining a parametric hand model MANO [74], with the mesh based representation for the object. As data for hand-object interaction is limited, we opt to use their synthetic dataset, the ObMan dataset [29], which is sufficiently large for training a neural network.…”

“…3.3. We compare with the baseline method [29] on the ObMan dataset and [28] on the FHB dataset. The results are summarized in Tab.…”

“…Due to many limiting factors of the real-world datasets such as FHB and HO3D (see Appendix B for detailed data analysis), learning a reasonable model for joint object and hand reconstruction is extremely challenging. Instead, to evaluate the effectiveness of the GF representation for 3D reconstruction on real-world images, we follow the setting of the latest work [28], where the object 3D model is given as input. Note that this is a com-monly used setting in previous works (e.g.…”

“…To analyse the effectiveness of the proposed grasping field representation for the 3D reconstruction task on real-world images, we compare our method with the latest work [28] on the FHB dataset. Compared to [29], the key difference in [28] is that the object is given as part of the input. We explore the same network architecture as [28] and only replace the decoder part with the grasping field.…”

“…Compared to [29], the key difference in [28] is that the object is given as part of the input. We explore the same network architecture as [28] and only replace the decoder part with the grasping field. The implementation details are presented in Appendix A (Fig.…”