Surface Capture for Performance-Based Animation

Abstract

Digital content production traditionally requires highly skilled artists and animators to first manually craft shape and appearance models and then instill the models with a believable performance. Motion capture technology is now increasingly used to record the articulated motion of a real human performance to increase the visual realism in animation. Motion capture is limited to recording only the skeletal motion of the human body and requires the use of specialist suits and markers to track articulated motion.In this paper we present surface capture, a fully automated system to capture shape and appearance as well as motion from multiple video cameras as a basis to create highly realistic animated content from an actor's performance in full wardrobe. We address wide-baseline scene reconstruction to provide 360 degree appearance from just 8 camera views and introduce an efficient scene representation for level of detail control in streaming and rendering. Finally we demonstrate interactive animation control in a computer games scenario using a captured library of human animation, achieving a frame rate of 300fps on consumer level graphics hardware. Index TermsImage-based modelling and rendering, Video-based character animation

Intrinsic Textures for Relightable Free-Viewpoint Video

“…The first two sequences [8] were recorded with 8 cameras at a resolution of 1920x1080, whereas the last sequence [24] was recorded with 11 cameras at 1296x972. In all renders a high level of detail is achieved, and challenging textures are faithfully reconstructed, as shown in Figure ??.…”

“…Recently, FVVR research has shifted from straightforward reproduction of the original scene to extending FVVR functionality [8,9] with the goal of adapting it to other applications. In particular, the ability to relight an actor's performance for seamless compositing into arbitrary real-world and computer-generated surroundings is highly desirable, and is termed Relightable Free-Viewpoint Video Rendering (RFVVR).…”

Colour Dynamic Photometric Stereo for Textured Surfaces

“…that project time-varying textures on the surface are rather difficult to apply on deformable surfaces. Passive techniques, such as conventional stereo or multicamera systems, are not biased by temporal variation, and they are more frequently applied for dynamic surfaces [1][2][3][4][5][6][7]. However, their accuracy is not sufficient to reconstruct small details, such as wrinkles and creases of garments.…”

Towards Plenoptic Raumzeit Reconstruction

“…One possibility is the incorporation of a geometry proxy. Given the input images a 3D representation for each discrete time step is reconstructed and used for depth guided resampling of the plenoptic function [17][18][19][20][21][22]. For restricted scene setups the incorporation of template models proves beneficial [23][24][25].…”