Dynamic Textures Synthesis as Nonlinear Manifold Learning and Traversing

Abstract

We formulate the problem of dynamic texture synthesis as a nonlinear manifold learning and traversing problem. We characterize dynamic textures as the temporal changes in spectral parameters of image sequences. For continuous changes of such parameters, it is commonly assumed that all these parameters lie on or close to a low-dimensional manifold embedded in the original configuration space. For complex dynamic data, the manifolds are usually nonlinear and we propose to use a mixture of linear subspaces to model a nonlinear manifold. These locally linear subspaces are further aligned within a global coordinate system. With the nonlinear manifold being globally parameterized, we overcome motion discontinuity problems encountered in switching linear models and dynamics. We present a nonparametric method to describe the complex dynamics of data sequences on the manifold. We also apply such approach to dynamic spatial parameters such as motion capture data. The experimental results suggest that our approach is able to synthesize smooth, complex dynamic textures and human motions, and has potential applications to other dynamic data synthesis problems.

Design of Beneficial Wave Dynamics for Engine Life and Operability Enhancement

“…Therefore, the distribution p is approximated by a finiite-dimensional vector u(x) € R m and let 48) where diag{u} is a diagonal matrix whose entries are the elements of the vector u. Similarly, the operators are approximated by their finite-dimensional matrices:…”

“…The model switching is determined by relative distances between the model's current state and the local means of neighboring data patches. The concept of nonlinear dynamic textures has appeared in [48,4,47,77]. An example of a piecewise affine dynamic texture model of the beach video sequence [4] is shown in Figure 1.17 (bottom row), where it is compared with a linear dynamic texture model (top row).…”

Learning Repetitive Patterns for Classifying Non-Rigidly Deforming Texture Surfaces

“…Nonlinear manifold learning is also a useful technique for the synthesis of dynamic textures. Liu et al [11] approach the dynamic texture synthesis problem using nonlinear manifold learning and traversing. In [6], Filipovych and Ribeiro proposed the use of non-linear manifold learning methods for modeling the texture deformations caused by surface curvature.…”

Low order dynamics embedding for high dimensional time series

“…Applications include, among others, appearance-based tracking [3,8], activity recognition from video data [4], gait recognition [1], video and image inpainting [15], and dynamic texture analysis and synthesis [10]. In all cases, the key to obtain computationally tractable algorithms is to exploit the high degree of correlation of the original data to obtain low dimensional representations, which are then used as surrogates to solve the problem of interest.…”