Modeling spatial and temporal variation in motion data

Abstract

Figure 1: First 3 columns: Results for cheering, walk cycle, and swimming motion. In each column, the top image shows the 4 inputs (overlapped, each with different color) and the bottom image shows the 15 outputs (overlapped, each with different color). These are frames from the animations. Please see the animations in the video. Last column: Results for 2D handwritten characters "a" and "2". Each image shows both the 4 inputs (blue) and 15 outputs (green). AbstractWe present a novel method to model and synthesize variation in motion data. Given a few examples of a particular type of motion as input, we learn a generative model that is able to synthesize a family of spatial and temporal variants that are statistically similar to the input examples. The new variants retain the features of the original examples, but are not exact copies of them. We learn a Dynamic Bayesian Network model from the input examples that enables us to capture properties of conditional independence in the data, and model it using a multivariate probability distribution. We present results for a variety of human motion, and 2D handwritten characters. We perform a user study to show that our new variants are less repetitive than typical game and crowd simulation approaches of re-playing a small number of existing motion clips. Our technique can synthesize new variants efficiently and has a small memory requirement.

Human 3D Motion Recognition Based on Spatial-Temporal Context of Joints

“…To validate the proposed ST-HMM for human action recognition, we tested it on a large database of 3D mocap data, obtained from the CMU Mocap database which has been used in similar work [4] [3]. It consists of 11 action classes; jump, run, crawl, climb ladders, stand up, sit down, and four types of swimming styles, e.g., back stroke, butterfly stroke, breast stroke and front crawl.…”

“…The temporal dependency in a human motion sequence is due to the strong temporal correlations of the 3D motion data in adjacent time-frames. There are several approaches that exploit the temporal dependency in the classification and recognition of motion patterns, e.g., [1] [2] [4] [3]. In our ST-HMM approach, the temporal dependency is captured along the horizontal direction in the state transition graph (st-graph) as shown in Fig1b.…”

“…For example, the position of the hip can be readily inferred from the position of the knee. A recent work by [4] adopts Dynamic Bayesian Network (DBN) to model the spatial and temporal variation of motion from a small number of visually similar motion clips. The structure of DBN represents the dependency relationship that underlies the spatial and temporal context.…”

Linguistics as structure in computer animation

“…Although there is ongoing research in using mocap data to support naturalistic motion synthesis (Lau et al 2009), motion capture still shares some of the same inflexibility as video recording. In an empirical study that compared the efficacy of motion capture and hand animation for portraying fingerspelling, viewers preferred the fingerspelling produced through hand animation (Sanders et al 2005).…”

Modeling head-related transfer functions via spatial-temporal Gaussian process

“…The key components of these models involve the specification of a distribution over a smooth random function (surface) with its mean surface representing the unknown HRTFs or HRIRs over a mesh of incident angles for a given anatomy configuration. In approaching the design of such a model, one must consider the spatial and temporal dependence features of the response: it is in this respect that we differ from the standard approach to such a problem, which involves separating the spatial and temporal dependences through a product space formulation, which is often common in machine learning and various applications of GP modeling, e.g., motion tracking modeling [14], modeling gas distribution [15], environmental surveillance [16], modeling MRI brain images [17], transcriptional landscape estimation [18], clustering gene expression [19], inter atomic potential models [20], and modeling of wire-cut electrical discharge machining(WEDM) [21] as discussed in [22,23]. In this paper, we consider the temporal and spatial features (co-variates) jointly in the covariance and mean functions.…”