Correspondence labelling for wide-timeframe free-form surface matching

Abstract

Abstract

Temporal Surface Tracking Using Mesh Evolution

“…In [8] this is used within a variational framework to fully track surfaces using level sets. As noticed in [12], flow-based approaches are nevertheless limited to small displacements, as a consequence of finite difference approximations of derivatives.…”

“…However, these approaches do not easily extend to real objects' surfaces recovered from images since the associated meshes can vary drastically between successive frames and furthermore, their topologies can change. In that case, photometric cues are advantageously added to geometric features to make the labeling feasible as in [12]. While allowing for large motions, labeling approaches seek dense correspondences, which are difficult to obtain on a regular basis over long sequences.…”

“…To this purpose, we build on some of the ideas already used in the approaches mentioned above. We combine the interest of both photometric and geometric cues for robust matching [12] with Laplacian diffusion [9] to get a dense displacement field. The resulting vector field is used to initialize a consistent mesh evolution between successive frames.…”

“…However, the associated differential methods are limited to small displacements between successive frames. Alternatively, feature-based approaches [10,11,12] consider meshes and allow for larger motions by casting the problem as a labeling between 2 meshes using local geometric or photometric information. This labeling solves for partial correspondences between 2 frames only and might lose efficiency when applied over long sequences, in particular as topological changes occur.…”

Dense 3D motion capture from synchronized video streams

“…Second, local motions are typically estimated independently between adjacent frames, then concatenated into long trajectories, causing accumulating drift [20] which may pose problems in applications such as body and face motion capture, or facial expression transfer from human actors to imaginary creatures [15,2]. A strategy aptly called "track to first" in [3] solves the accumulation problem, and it is exploited in our approach (see [5,22] for approaches free from accumulation drifts).…”