Interaction-Based Human Activity Comparison

Abstract

Traditional methods for motion comparison consider features from individual characters. However, the semantic meaning of many human activities is usually defined by the interaction between them, such as a high-five interaction of two characters. There is little success in adapting interaction-based features in activity comparison, as they either do not have a fixed topology or are in high dimensional. In this paper, we propose a unified framework for activity comparison from the interaction point of view. Our new metric evaluates the similarity of interaction by adapting the Earth Mover's Distance onto a customized geometric mesh structure that represents spatial-temporal interactions. This allows us to compare different classes of interactions and discover their intrinsic semantic similarity. We created five interaction databases of different natures, covering both twocharacters (synthetic and real-people) and character-object interactions, which are open for public uses. We demonstrate how the proposed metric aligns well with the semantic meaning of the interaction. We also apply the metric in interaction retrieval and show how it outperforms existing ones. The proposed method can be used for unsupervised activity detection in monitoring systems and activity retrieval in smart animation systems.

3D car shape reconstruction from a contour sketch using GAN and lazy learning

“…We then register the point clouds from different cars by considering this process as an Earth Mover problem [19,46]. This means that we will first select one point cloud randomly Fig.…”

Automatic Sign Dance Synthesis from Gesture-based Sign Language

“…Hyun et al [17] presented a set of high-level language known as motion grammar to describe interaction for multi-character interaction synthesis. Shen et al [28] demonstrated a mesh structure to understand multi-character interaction. In this work, we consider the correlation between dance and sign gesture for a virtual characters.…”

Prior-less 3D Human Shape Reconstruction with an Earth Mover’s Distance Informed CNN

“…Such a comparison is required when we build the reconstruction loss function. Motivated by [2,5], we propose the use of EMD in our deep learning loss. EMD evaluates the minimum overall distance between two point clouds by finding the optimal mapping between them.…”

“…In order to facilitate the use of an unordered point cloud, we need to be able to measure the distance between the reconstructed points and the ground truths. Motivated by [2,5], we propose a novel loss function that incorporates the Earth Mover's Distance (EMD). This method determines the optimal alignment between two point cloud distributions, and allows for evaluation of the reconstruction accuracy.…”

“…• An end-to-end deep learning framework for 3D human shape reconstruction from a 2D image without the need for a 3D prior parametric model. • A novel loss function based upon EMD [5] to evaluate the distances between unordered 3D point clouds representing human body shapes. • A synthetic pairwise 2D and 3D dataset to train our deep learning framework inspired by [6].…”