Intrinsic Characterization of Dynamic Surfaces

Tung, Tony; Matsuyama, Takashi

doi:10.1109/cvpr.2013.37

Cited by 4 publications

(13 citation statements)

References 44 publications

(84 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Collected information is usually represented as a stream of surface mesh models undergoing free-form deformation across time, which geometrical structure (e.g., surface mesh connectivity) can be kept consistent using 3D scene flow estimation, or surface-point matching and tracking [10,26,3,42,17]. Hence, low-frequency surface details (e.g., wrinkles on solid color clothing) can be tracked accurately, and deformation dynamics can be characterized for various classification tasks [43].…”

Section: Related Workmentioning

confidence: 99%

“…State-of-the-art dynamic event models usually consider sets of independent features sparsely distributed in space [4,38] or in time [24,43]. However, natural scenes (e.g., in 3D video or dynamic texture) often exhibit both time and space dependent observations: a point P(t) at time t is correlated to observations of P(t + 1) and P (t), where P (t) belongs to the (spatial) neighborhood V(P(t)) of P(t).…”

Section: Descriptor Spatial Layoutmentioning

confidence: 99%

“…We introduce the bag-of-timings (BoT) paradigm for dynamic surface classification, by treating timings of local surface element (i.e., keypoint) dynamics as words to be represented in a codebook, while in previous work LDS state parameters were used as words (i.e., not timing structure gaps) [33,43]. Bag-of-words (BoW) are also used for image classification where image features serve as words [15].…”

Section: Bag-of-timingsmentioning

confidence: 99%

“…In [43], the authors demonstrated that surface rigidity can be characterized by modeling intrinsic property dynamics using linear dynamical systems (LDS), which have long history in dynamics modeling (see dynamic textures [34,11,12,4,44,33]). However their model only accounts for system state duration of independent observations, and is so far limited to binary classification.…”

Section: Introductionmentioning

confidence: 99%

“…Complex surface deformations can be modeled using sets of LDS where observations across time are given by surface intrinsic property variations such as shape indices [43].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Timing-Based Local Descriptor for Dynamic Surfaces

Tung

Matsuyama

2014

2014 IEEE Conference on Computer Vision and Pattern Recognition

Self Cite

View full text Add to dashboard Cite

In this paper, we present the first local descriptor designed for dynamic surfaces. A dynamic surface is a surface that can undergo non-rigid deformation (e.g., human body surface). Using state-of-the-art technology, details on dynamic surfaces such as cloth wrinkle or facial expression can be accurately reconstructed. Hence, various results (e.g., surface rigidity, or elasticity) could be derived by microscopic categorization of surface elements. We propose a timing-based descriptor to model local spatiotemporal variations of surface intrinsic properties. The lowlevel descriptor encodes gaps between local event dynamics of neighboring keypoints using timing structure of linear dynamical systems (LDS). We also introduce the bagof-timings (BoT) paradigm for surface dynamics characterization. Experiments are performed on synthesized and real-world datasets. We show the proposed descriptor can be used for challenging dynamic surface classification and segmentation with respect to rigidity at surface keypoints.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Descriptor Spatial Layoutmentioning

confidence: 99%

Section: Bag-of-timingsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…Complex surface deformations can be modeled using sets of LDS where observations across time are given by surface intrinsic property variations such as shape indices [43].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Timing-Based Local Descriptor for Dynamic Surfaces

Tung

Matsuyama

2014

2014 IEEE Conference on Computer Vision and Pattern Recognition

Self Cite

View full text Add to dashboard Cite

show abstract

DeepWrinkles: Accurate and Realistic Clothing Modeling

Lähner

Cremers

Tung

2018

Lecture Notes in Computer Science

Self Cite

189

176

View full text Add to dashboard Cite

We present a novel method to generate accurate and realistic clothing deformation from real data capture. Previous methods for realistic cloth modeling mainly rely on intensive computation of physicsbased simulation (with numerous heuristic parameters), while models reconstructed from visual observations typically suffer from lack of geometric details. Here, we propose an original framework consisting of two modules that work jointly to represent global shape deformation as well as surface details with high fidelity. Global shape deformations are recovered from a subspace model learned from 3D data of clothed people in motion, while high frequency details are added to normal maps created using a conditional Generative Adversarial Network whose architecture is designed to enforce realism and temporal consistency. This leads to unprecedented high-quality rendering of clothing deformation sequences, where fine wrinkles from (real) high resolution observations can be recovered. In addition, as the model is learned independently from body shape and pose, the framework is suitable for applications that require retargeting (e.g., body animation). Our experiments show original high quality results with a flexible model. We claim an entirely data-driven approach to realistic cloth wrinkle generation is possible.

show abstract

On Mean Pose and Variability of 3D Deformable Models

Allain

Franco

Boyer

et al. 2014

Computer Vision – ECCV 2014

Self Cite

View full text Add to dashboard Cite

International audienceWe present a novel methodology for the analysis of complex object shapes in motion observed by multiple video cameras. In particular, we propose to learn local surface rigidity probabilities (i.e., deformations), and to estimate a mean pose over a temporal sequence. Local deformations can be used for rigidity-based dynamic surface segmentation, while a mean pose can be used as a sequence keyframe or a cluster prototype and has therefore numerous applications, such as motion synthesis or sequential alignment for compression or morphing. We take advantage of recent advances in surface tracking techniques to formulate a generative model of 3D temporal sequences using a probabilistic framework, which conditions shape fitting over all frames to a simple set of intrinsic surface rigidity properties. Surface tracking and rigidity variable estimation can then be formulated as an Expectation-Maximization inference problem and solved by alternatively minimizing two nested fixed point iterations. We show that this framework provides a new fundamental building block for various applications of shape analysis, and achieves comparable tracking performance to state of the art surface tracking techniques on real datasets, even compared to approaches using strong kinematic priors such as rigid skeletons

show abstract

Intrinsic Characterization of Dynamic Surfaces

Cited by 4 publications

References 44 publications

Timing-Based Local Descriptor for Dynamic Surfaces

Timing-Based Local Descriptor for Dynamic Surfaces

DeepWrinkles: Accurate and Realistic Clothing Modeling

On Mean Pose and Variability of 3D Deformable Models

Contact Info

Product

Resources

About