Shape2Pose

Abstract

As 3D acquisition devices and modeling tools become widely available there is a growing need for automatic algorithms that analyze the semantics and functionality of digitized shapes. Most recent research has focused on analyzing geometric structures of shapes. Our work is motivated by the observation that a majority of manmade shapes are designed to be used by people. Thus, in order to fully understand their semantics, one needs to answer a fundamental question: "how do people interact with these objects?" As an initial step towards this goal, we offer a novel algorithm for automatically predicting a static pose that a person would need to adopt in order to use an object. Specifically, given an input 3D shape, the goal of our analysis is to predict a corresponding human pose, including contact points and kinematic parameters. This is especially challenging for man-made objects that commonly exhibit a lot of variance in their geometric structure. We address this challenge by observing that contact points usually share consistent local geometric features related to the anthropometric properties of corresponding parts and that human body is subject to kinematic constraints and priors. Accordingly, our method effectively combines local region classification and global kinematically-constrained search to successfully predict poses for various objects. We also evaluate our algorithm on six diverse collections of 3D polygonal models (chairs, gym equipment, cockpits, carts, bicycles, and bipedal devices) containing a total of 147 models. Finally, we demonstrate that the poses predicted by our algorithm can be used in several shape analysis problems, such as establishing correspondences between objects, detecting salient regions, finding informative viewpoints, and retrieving functionally-similar shapes.

Data-driven shape analysis and processing

“…Thus, analyzing shapes and scenes requires some understanding of their function. The recent work by Laga et al [LMS13] and Kim et al [KCGF14] are important examples of data-driven approaches that take into account functional aspects in shape analysis. In addition, data-driven methods can guide the synthesis of shapes that can be manufactured or 3D printed based on given functional specifications; an example of such attempt is the work by Schulz et al [ Feature learning.…”

Inferring object properties from human interaction and transferring them to new motions

“…A variety of techniques in shape analysis have been developed to extract functional information about objects and scenes using human-object interaction as cues. An appropriate human pose or action map can be created from an input object [11][12][13] or scene [14,15]; see the survey in Ref. [16] for more information.…”

Character-Object Interaction Retrieval using the Interaction Bisector Surface

“…For example, Grabner et al [GGVG11] build a Gaussian model to represent the spatial relationship between the character and a chair, based on the distance and intersection between the polygon meshes representing the character and the chair. Kim et al [KCGF14] predict human pose based on the trained affordance model. Jiang et al [JKS13] use Euclidean distance, relative angle and height distance to represent the relationship between character and point cloud environment.…”