Cage-Based Motion Recovery Using Manifold Learning

Abstract

We present a flexible model-based approach for the recovery of parameterized motion from a sequence of 3D meshes without temporal coherence. Unlike previous model-based approaches using skeletons, we embed the deformation of a reference mesh template within a low polygonal representation of the mesh, namely the cage, using Green Coordinates. The advantage is a less constrained model that more robustly adapts to noisy observations while still providing structured motion information, as required by several applications. The cage is parameterized with a set of 3D features dedicated to the description of human morphology. This allows to formalize a novel representation of 3D meshed and articulated characters, the Oriented Quads Rigging (OQR). To regularize the tracking, the OQR space is subsequently constrained to plausible poses using manifold learning. Results are shown for sequences of meshes, with and without temporal coherence, obtained from multiple view videos preprocessed by visual hull. Motion recovery applications are illustrated with a motion transfer encoding and the extraction of trajectories of anatomical joints. Validation is performed on the HumanEva II database.

Human Shape and Pose Tracking Using Keyframes

“…In a non-sequential strategy, Budd et al [4] and Klaudiny et al [14] assume that the complete input sequence is available beforehand and they find the best order to traverse it using a minimum spanning tree algorithm. Duveau et al [8] propose a supervised learning strategy that regularizes the results based on the learned distribution in a latent parameter space. These methods require a pre-processing step either to build a shape-similarity tree from the input sequence [4,14], or to learn a lowdimensional representation from the gathered motion training data [8,19].…”

“…Duveau et al [8] propose a supervised learning strategy that regularizes the results based on the learned distribution in a latent parameter space. These methods require a pre-processing step either to build a shape-similarity tree from the input sequence [4,14], or to learn a lowdimensional representation from the gathered motion training data [8,19]. Our multiple-keyframe approach also exploits temporal information.…”

3D Visualization of Body Motion in Speed Climbing

“…Instead, we use the set of 3D meshes in motion to learn a manifold of 3D shapes of the athlete. The tracking procedure consists in registering the 3D mesh model of the athlete onto the two drones video view by optimizing the manifold parameters with respect to salient body features (Duveau et al, 2012 ). The manifold allows for a reduction in dimensions which guarantees the convergence of the registration.…”

“…We focus on a specific athlete for which we built a dedicated 3D biomechanical twin, or avatar. Using this model, we adapted one of our previous work on manifold learning of 3D body shapes in motion (Duveau et al, 2012 ) to speed climbing gesture.…”

Adaptive skeleton‐driven cages for mesh sequences

“…As a result, the reproduced sequence is at high fidelity compared with that obtained in . Duveau et al addressed the cage sequence recovery problem for raw captured data via the machine learning method. However, intuitive cage manipulation is still not addressed by the methods described previously.…”