Understanding and Exploiting Object Interaction Landscapes

Abstract

Interactions play a key role in understanding objects and scenes for both virtual and real-world agents. We introduce a new general representation for proximal interactions among physical objects that is agnostic to the type of objects or interaction involved. The representation is based on tracking particles on one of the participating objects and then observing them with sensors appropriately placed in the interaction volume or on the interaction surfaces. We show how to factorize these interaction descriptors and project them into a particular participating object so as to obtain a new functional descriptor for that object, its interaction landscape , capturing its observed use in a spatiotemporal framework. Interaction landscapes are independent of the particular interaction and capture subtle dynamic effects in how objects move and behave when in functional use. Our method relates objects based on their function, establishes correspondences between shapes based on functional key points and regions, and retrieves peer and partner objects with respect to an interaction.

Placement Retargeting of Virtual Avatars to Dissimilar Indoor Environments

“…On the object interaction level, researchers have developed various methods to represent and estimate object affordance [21] or functionality, e.g., [22], [23], [24]. Among them, Hu et al [25] proposed a geometric description of the functionality of a 3D object in the context of a given scene, derived from interactions between objects.…”

Geometric Affordance Perception: Leveraging Deep 3D Saliency With the Interaction Tensor

“…Previous methods employing the IBS for functionality analysis in Zhao et al (2014), Hu et al (2015, Zhao et al (2016), Pirk et al (2017) have relied on the computation of local shape features on the bisector surface, these features are then fed to a machine learning algorithm (or similarity function) in order to retrieve or synthesize similar interactions. Then, for any new pair of objects, they need first to perform many costly computations to make a prediction; namely, estimate the IBS with dense pointclouds and remove noisy data, compute shape and topological features at various locations using mesh information, and (typically) build histograms representing the global shape features.…”

3D Functionality Analysis for Shape Modeling via Partial Matching

“…Furthermore, the probabilistic model proposed by Savva et al [52] encodes the interactions between a human pose and objects in a scene into a set of prototypical interaction graphs (PiGraphs), which is a human-centric representation of interactions that captures physical contacts and visual attention linkages between the human pose and 3D geometry. Instead of using a specific agent, Pirk et al [47] introduced a more general means of understanding object interactions. In their work, a descriptor characterizes proximal interactions between a target object and a motion driver, where motion driver refers to anything that interacts with a static object, including an agent such as a human pose, also including other objects such as a human hand or wind.…”

“…However, in consequence, the generated hybrid shapes can be hardly categorized into known functional categories. This brings a new challenge: to adapt and enhance functionality models which were designed for discriminative analysis of single categories [23,25,47] to serve shape modeling. In particular, we must address the issue that new cross-category hybrid shapes may arise whose functionality models cannot be learned in advance.…”