Deformable object tracking using optical flow constraints

Abstract

In this paper, we present a method for deformable object tracking that exploits the entire image information using the optical flow equation instead of working with discrete feature points. Our method starts from the optical flow constraint and first estimates global transformations in a hierarchical framework. Elastic deformations are then estimated separately using deformable meshes and spatial and temporal smoothing constraints. In both cases additional constraints to regularize the optical flow field are obtained from the motion model such that finding the best transformation amounts in minimizing an error function that can be solved in a least squares sense. Combing deformable meshes and the optical flow equation with a dedicated weighted smoothness constraint on the mesh deformation and estimating global transformations separately from elastic deformations is key to dealing with complex deformations such as cloth deformation as a person moves.

Optical flow based tracking and retexturing of garments

“…However, the tracking method described in Section 2 is not limited to the dot pattern used here. It can also be applied to arbitrary logos on shirts that are rich enough in detail to exploit optical flow [13], [15] (see Fig. 1).…”

“…E(d k ) is a measure of vertex displacement deviation to the displacements of its neighbors. Details on the smoothing constraints can be found in [13]. The weight w k can be e.g.…”

“…In [9] Scholz et al presented an optical flow based multicamera system where optical flow tracking is combined with silhouette matching. In [13] we demonstrated a method to track deformations of a shirt in a pre-segmented binary video sequence using the optical flow and deformable meshes with weighted temporal and spatial smoothing constraints.…”

Designed to Fit: Challenges of Interaction Design for Clothes Fitting Room Technologies

“…Virtual Fittings: Some online services, such as Intellifit [14] and MyShape [19], take measurements of a customer's body and suggest clothes that will fit and provide a flattering appearance for that body shape. These services have been deployed in stores such as Levi's and Charming Shoppes in Ohio, Florida and Texas [34].…”