Motion Words: A Text-Like Representation of 3D Skeleton Sequences

Abstract

There is a growing amount of human motion data captured as a continuous 3D skeleton sequence without any information about its semantic partitioning. To make such unsegmented and unlabeled data efficiently accessible, we propose to transform them into a text-like representation and employ well-known text retrieval models. Specifically, we partition each motion synthetically into a sequence of short segments and quantize the segments into motion words, i.e. compact features with similar characteristics as words in text documents. We introduce several quantization techniques for building motion-word vocabularies and propose application-independent criteria for assessing the vocabulary quality. We verify these criteria on two real-life application scenarios.

Content-Based Management of Human Motion Data: Survey and Challenges

“…However, there are also alternative approaches that cut the unsegmented data into a higher number of short overlapping segments, represent these by very compact features, and then compare sequences of such features. To obtain the compact features, the short segments are first represented by arbitrary high-dimensional features (e.g., raw skeleton data in [39] or deep features in [19]), then the space of the segment features is clustered, and cluster identifiers are used to form a vocabulary (codebook). Individual segments are then represented by the one-dimensional identifiers of the closest cluster, so the similarity of two short segments is reduced to a trivial equality over the quantized features.…”

“…Individual segments are then represented by the one-dimensional identifiers of the closest cluster, so the similarity of two short segments is reduced to a trivial equality over the quantized features. The skeleton sequences can be then represented by sequences [39], histograms [19], [68], [69] or bags [40] of the quantized features. The bag-of-words representation proposed in [40] is mainly interesting from the large-scale processing perspective, since it enables application of efficient and scalable text-retrieval techniques (e.g., inverted files) for a variety of motion processing tasks.…”

“…Compact representations reduce the processing costs dramatically but introduce the risk of oversimplification that leads to the deterioration of distinctiveness between originally dissimilar motions. For this reason, compact features (e.g., motion words [39], motifs [19], or signatures [38]) are more suitable for the pre-search phase, whereas the high-dimensional features (e.g., deep features [36] or raw 3D coordinates [44]) for the more expensive refinement phase.…”

“…State-of-the-art deep features are typically high-dimensional vectors, which are known to be difficult to index due to the curse of dimensionality [120]. Therefore, researchers should continue to look for indexable motion features that provide a reasonable trade-off between feature expressiveness and its complexity [39]. The indexable features would be used for fast identification of candidate motions, which could be then refined using more precise and costly similarity evaluations.…”

Efficient Indexing of 3D Human Motions

“…The efficiencyoriented works either propose very compact features that allow fast sequential scanning [12,13], or utilize various indexing schemes to organize the motion data (e.g., the binary tree [25], kd tree [9], R* tree [4], inverted file index [14], or tries [8]). To optimize the efficiency-effectiveness trade-off, a two-phase retrieval model is often used, where the candidate objects identified within an efficient search phase are submitted to a re-ranking phase that refines the result using more expensive techniques (e.g., traversal of a graph structure [9] or ranking by the Dynamic Time Warping [14,20]). A more thorough discussion and comparison of all these methods can be found in the recent survey [21].…”