High-quality passive facial performance capture using anchor frames

Beeler, Thabo; Hahn, Fabian; Bradley, Derek; Bickel, Bernd; Beardsley, Paul; Gotsman, Craig; Sumner, Robert W.; Groß, Markus

doi:10.1145/1964921.1964970

Cited by 127 publications

(109 citation statements)

References 25 publications

Supporting

Mentioning

108

Contrasting

Order By: Relevance

“…This problem is eliminated with the non- (frames 0, 121, 160, 195, 240, 260, 341, 354) Fig. 9 Comparison of non-sequential alignment using anchor frames of a single expression (Beeler et al 2011) and minimum spanning tree (data courtesy Beeler et al Disney Research) sequential alignment where the pattern remains accurately aligned with the face throughout the sequence.…”

Section: Results Of Global Alignmentmentioning

confidence: 99%

“…A comparison of facial alignment using the anchor frame approach of Beeler et al (2011) to our non-sequential approach is presented in Fig. 9 using publicly available datasets.…”

Section: Results Of Global Alignmentmentioning

confidence: 99%

“…This non-sequential reconstruction improves efficiency and reduces accumulation of reconstruction errors across the sequence. Beeler et al (2011) presented an approach based on anchor frames to reduce drift for alignment of reconstructed non-rigid face sequences. The approach assumes anchor frames similar to a manually selected reference expression are distributed across the sequence.…”

Section: Non-sequential Non-rigid Surface Alignmentmentioning

confidence: 99%

“…Pairwise non-rigid alignment will result in drift as the alignment is propagated across the tree resulting in accumulation of errors. Drift can potentially be alleviated by alignment to a reference mesh (Furukawa and Ponce 2010) or anchor frames (Beeler et al 2011). However, there will be an accumulation of errors as the number of intermediate frames and path length across the tree increases.…”

Section: Multi-path Non-sequential Alignmentmentioning

confidence: 99%

See 3 more Smart Citations

Global Non-rigid Alignment of Surface Sequences

et al. 2012

View full text Add to dashboard Cite

This paper presents a general approach based on the shape similarity tree for non-sequential alignment across databases of multiple unstructured mesh sequences from non-rigid surface capture. The optimal shape similarity tree for non-rigid alignment is defined as the minimum spanning tree in shape similarity space. Non-sequential alignment based on the shape similarity tree minimises the total non-rigid deformation required to register all frames in a database into a consistent mesh structure with surfaces in correspondence. This allows alignment across multiple sequences of different motions, reduces drift in sequential alignment and is robust to rapid non-rigid motion. Evaluation is performed on three benchmark databases of 3D mesh sequences with a variety of complex human and cloth motion. Comparison with sequential alignment demonstrates reduced errors due to drift and improved robustness to large non-rigid deformation, together with global alignment across multiple sequences which is not possible with previous sequential approaches.

show abstract

Section: Results Of Global Alignmentmentioning

confidence: 99%

“…A comparison of facial alignment using the anchor frame approach of Beeler et al (2011) to our non-sequential approach is presented in Fig. 9 using publicly available datasets.…”

Section: Results Of Global Alignmentmentioning

confidence: 99%

Section: Non-sequential Non-rigid Surface Alignmentmentioning

confidence: 99%

Section: Multi-path Non-sequential Alignmentmentioning

confidence: 99%

See 2 more Smart Citations

Global Non-rigid Alignment of Surface Sequences

et al. 2012

View full text Add to dashboard Cite

show abstract

“…Representative techniques include three-dimensional (3D) model-based methods and twodimensional (2D) image-based methods. The 3D model-based methods include blendshapes with several shape models [12], [22] and expression retargeting with a motion-capture system [3], [19]. The 2D image-based methods synthesize mouth animations using a prepared video corpus [5], [8].…”

Section: Introductionmentioning

confidence: 99%

Automatic Generation of Photorealistic 3D Inner Mouth Animation only from Frontal Images

Kawai

Iwao

Maejima

et al. 2015

Journal of Information Processing

View full text Add to dashboard Cite

Abstract:In this paper, we propose a novel method to generate highly photorealistic three-dimensional (3D) inner mouth animation that is well-fitted to an original ready-made speech animation using only frontal captured images and small-size databases. The algorithms are composed of quasi-3D model reconstruction and motion control of teeth and the tongue, and final compositing of photorealistic speech animation synthesis tailored to the original. In general, producing a satisfactory photorealistic appearance of the inner mouth that is synchronized with mouth movement is a very complicated and time-consuming task. This is because the tongue and mouth are too flexible and delicate to be modeled with the large number of meshes required. Therefore, in some cases, this process is omitted or replaced with a very simple generic model. Our proposed method, on the other hand, can automatically generate 3D inner mouth appearances by improving photorealism with only three inputs: an original tailor-made lip-sync animation, a single image of the speaker's teeth, and a syllabic decomposition of the desired speech. The key idea of our proposed method is to combine 3D reconstruction and simulation with two-dimensional (2D) image processing using only the above three inputs, as well as a tongue database and mouth database. The satisfactory performance of our proposed method is illustrated by the significant improvement in picture quality of several tailor-made animations to a degree nearly equivalent to that of camera-captured videos.

show abstract