Human video textures

Flagg, Matthew; Nakazawa, Atsushi; Zhang, Qiushuang; Kang, Sing Bing; Ryu, Young Kee; Essa, Irfan; Rehg, James M.

doi:10.1145/1507149.1507182

Cited by 39 publications

(37 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Transition frames were manually identified in each input clip for the purpose of video duration extension via concatenation at transitions. However, it is possible to generate a greater variety of video-based pedestrian animations over a motion graph [5]. The video results illustrate the appearance of crowds constructed using crowd tubes before and after satisfying constraints, thus demonstrating the perceptual effect of increasing crowd density by relaxing collision and variety constraints.…”

Section: Summary Of Approachmentioning

confidence: 93%

“…In the spirit of video textures, we assume that infinite playback of V i is possible by computing transition points within V i and walking the motion graph comprised of a node per frame and a directed edge per transition. Recent work [5] presented an in-studio approach to generating human video textures, or controllable animations made from joint video and motion capture of human motion. For the purpose of investigating the constrained layout challenges facing video-based crowd synthesis, we assume that each V i has been preprocessed to identify transition frames and the motion graph is constrained to looping.…”

Section: Behavior and Density Controlmentioning

confidence: 99%

See 1 more Smart Citation

Video-Based Crowd Synthesis

Flagg

Rehg

2013

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Abstract-As a controllable medium, video-realistic crowds are important for creating the illusion of a populated reality in special effects, games, and architectural visualization. While recent progress in simulation and motion captured-based techniques for crowd synthesis has focused on natural macroscale behavior, this paper addresses the complementary problem of synthesizing crowds with realistic microscale behavior and appearance. Example-based synthesis methods such as video textures are an appealing alternative to conventional model-based methods, but current techniques are unable to represent and satisfy constraints between video sprites and the scene. This paper describes how to synthesize crowds by segmenting pedestrians from input videos of natural crowds and optimally placing them into an output video while satisfying environmental constraints imposed by the scene. We introduce crowd tubes, a representation of video objects designed to compose a crowd of video billboards while avoiding collisions between static and dynamic obstacles. The approach consists of representing crowd tube samples and constraint violations with a conflict graph. The maximal independent set yields a dense constraint-satisyfing crowd composition. We present a prototype system for the capture, analysis, synthesis, and control of video-based crowds. Several results demonstrate the system's ability to generate videos of crowds which exhibit a variety of natural behaviors.

show abstract

Section: Summary Of Approachmentioning

confidence: 93%

Section: Behavior and Density Controlmentioning

confidence: 99%

Video-Based Crowd Synthesis

Flagg

Rehg

2013

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

show abstract

“…A 2D shape template is then fitted onto the character in the input image to drive the deformation according to projected 3D motion data. Flagg et al [Flagg et al 2009] also exploited the combination of 2D video and skeletal MoCap to generate controllable animations of human performance. 2D video and MoCap data are first synchronised.…”

Section: Related Workmentioning

confidence: 99%

Hybrid Skeletal-Surface Motion Graphs for Character Animation from 4D Performance Capture

et al. 2015

View full text Add to dashboard Cite

We present a novel hybrid representation for character animation from 4D Performance Capture (4DPC) data which combines skeletal control with surface motion graphs. 4DPC data are temporally aligned 3D mesh sequence reconstructions of the dynamic surface shape and associated appearance from multiple view video. The hybrid representation supports the production of novel surface sequences which satisfy constraints from user specified key-frames or a target skeletal motion. Motion graph path optimisation concatenates fragments of 4DPC data to satisfy the constraints whilst maintaining plausible surface motion at transitions between sequences. Spacetime editing of the mesh sequence using a learnt part-based Laplacian surface deformation model is performed to match the target skeletal motion and transition between sequences. The approach is quantitatively evaluated for three 4DPC datasets with a variety of clothing styles. Results for key-frame animation demonstrate production of novel sequences which satisfy constraints on timing and position of less than 1% of the sequence duration and path length. Evaluation of motion capture driven animation over a corpus of 130 sequences shows that the synthesised motion accurately matches the target skeletal motion. The combination of skeletal control with the surface motion graph extends the range and style of motion which can be produced whilst maintaining the natural dynamics of shape and appearance from the captured performance.

show abstract

“…Their system was demonstrated on several examples including a mouse controlled fish, whereby a mouse cursor was used to guide the path of the fish with different velocities. Similarly, Flagg et al [20] presented Human Video Textures, where, given a video of a martial artist performing various actions, they produce a photorealistic avatar which can be controlled, akin to a combat game character. Lee et al [21] used interactive controllers to animate an avatar from human motion captured data.…”

Section: Related Workmentioning

confidence: 99%

MIMiC: Multimodal Interactive Motion Controller

Okwechime

Ong

Bowden

2011

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Abstract-We introduce a new algorithm for real-time interactive motion control and demonstrate its application to motion captured data, pre-recorded videos and HCI. Firstly, a data set of frames are projected into a lower dimensional space. An appearance model is learnt using a multivariate probability distribution. A novel approach to determining transition points is presented based on k-medoids, whereby appropriate points of intersection in the motion trajectory are derived as cluster centres. These points are used to segment the data into smaller subsequences. A transition matrix combined with a kernel density estimation is used to determine suitable transitions between the subsequences to develop novel motion. To facilitate real-time interactive control, conditional probabilities are used to derive motion given user commands. The user commands can come from any modality including auditory, touch and gesture. The system is also extended to HCI using audio signals of speech in a conversation to trigger non-verbal responses from a synthetic listener in real-time. We demonstrate the flexibility of the model by presenting results ranging from data sets composed of vectorised images, 2D and 3D point representations. Results show real-time interaction and plausible motion generation between different types of movement.

show abstract

Human video textures

Cited by 39 publications

References 18 publications

Video-Based Crowd Synthesis

Video-Based Crowd Synthesis

Hybrid Skeletal-Surface Motion Graphs for Character Animation from 4D Performance Capture

MIMiC: Multimodal Interactive Motion Controller

Contact Info

Product

Resources

About