Modeling spatial relations of human body parts for indexing and retrieving close character interactions

Abstract

Retrieving pre-captured human motion for analyzing and synthesizing virtual character movement have been widely used in Virtual Reality (VR) and interactive computer graphics applications. In this paper, we propose a new human pose representation, called Spatial Relations of Human Body Parts (SRBP), to represent spatial relations between body parts of the subject(s), which intuitively describes how much the body parts are interacting with each other. Since SRBP is computed from the local structure (i.e. multiple body parts in proximity) of the pose instead of the information from individual or pairwise joints as in previous approaches, the new representation is robust to minor variations of individual joint location. Experimental results show that SRBP outperforms the existing skeleton-based motion retrieval and classification approaches on benchmark databases.

Interaction-Based Human Activity Comparison

“…There are some attempts to apply interaction mesh in interaction retrieval [29], [30], [31]. However, the results are not satisfying.…”

“…For the edges that co-exist in two interaction meshes, a traditional geometry-based distance function is applied. For those that do not co-exist due to the topological difference, [30] [31] utilizes an affinity matrix calculated based on a heuristic to extract the active joint pairs, but the heuristic requires domain knowledge and is likely dependent on the types of interaction.…”

A Two-Stream Recurrent Network for Skeleton-based Human Interaction Recognition

“…Modeling human-human interactions in competitive sports (such as kick-boxing [1]) and close interactions with tangling limbs (such as judo [2]) have benefited the computer animation community. Enhancing the motion features such as interaction mesh [2] demonstrate better results in motion retrieval [3], [4] and classification tasks [5]. More recently, the emergence of Recurrent Neural Network (RNN) enables automatically modeling the temporal dependency of motion sequences.…”

“…However, previous works in interaction recognition [7], [8] mainly focus on extracting features from individual characters that ignore the contextual information of the interaction be-tween characters. Another problem is the lack of exploring the correlations among joints, such as modeling body part relations [5], or plainly stacking the joint features [9], which causes the similar interactions less likely to be distinguished by such models. To this end, we propose a two-stream framework by exploring interaction representations under distance-based and joint-based features, respectively.…”

Illumination-Based Data Augmentation for Robust Background Subtraction

“…the static parts of a given scene) [1]. A large number of realworld applications, such as person re-identification [2], object tracking [3], gesture recognition [4], vehicle tracking [5], video recognition [6], action recognition [7], [8], crowd analysis [9] and even use cases of the medical domain [10], [11], depend on accurate and robust background subtraction as a first step of their pipelines.…”