Stitch it in Time: GAN-Based Facial Editing of Real Videos

Tzaban, Rotem; Mokady, Ron; Gal, Ran; Bermano, Amit H.; Cohen–Or, Daniel

doi:10.1145/3550469.3555382

Cited by 47 publications

(24 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As we demonstrate in the experiments, previous approaches have difficulty handling these challenging cases. With our GAN inversion, we can modify the latent code to perform high-quality semantic image editing [18,24,40,45] or video editing [53,58,60].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

In-N-Out: Face Video Inversion and Editing with Volumetric Decomposition

Xu¹,

Shu²,

Smith³

et al. 2023

Preprint

View full text Add to dashboard Cite

Section: Related Workmentioning

confidence: 99%

“…By changing the latent code, one can achieve many creative semantic editing effects [18,24,40,45] for images. For the video domain, recent methods also achieve faithful and temporally consistent editing [53,58,60]. However, these 2D methods often lack explicit viewpoint control of the generated contents.…”

Section: Introductionmentioning

confidence: 99%

In-N-Out: Face Video Inversion and Editing with Volumetric Decomposition

Xu¹,

Shu²,

Smith³

et al. 2023

Preprint

View full text Add to dashboard Cite

“…Other methods combine the optimization and encoder approaches and propose hybrid strategies by using the encoder for initialization and refining the latent code by optimization [35,95]. Recent 2D GAN inversion methods achieve faithful reconstruction with high editing capabilities and have been extended for video editing [2,75,88]. However, editing 3D-related attributes such as camera parameters and head pose remains inconsistent and prone to severe flickering as the pre-trained generator is unaware of the 3D structure.…”

Section: Gan Inversion Gan Inversion Maps a Real Image Backmentioning

confidence: 99%

DisCO: Portrait Distortion Correction with Perspective-Aware 3D GANs

Wang¹,

Liu²,

Huang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Specifically, video inversion should be temporally consistent. Tzaban et al [TMG*22] demonstrate that by combining encoders [TAN*21] with generator tuning techniques [RMBCO21], the consistency of the original video can be maintained. Another challenge can be found in the texture‐sticking phenomenon observed in StyleGAN1 and StyleGAN2 [KAL*21], which hinders the realism of generated and manipulated videos.…”

Section: Encoding and Inversionmentioning

confidence: 99%

State‐of‐the‐Art in the Architecture, Methods and Applications of StyleGAN

Bermano

Gal

Alaluf

et al. 2022

Computer Graphics Forum

View full text Add to dashboard Cite

Generative Adversarial Networks (GANs) have established themselves as a prevalent approach to image synthesis. Of these, StyleGAN offers a fascinating case study, owing to its remarkable visual quality and an ability to support a large array of downstream tasks. This state-of-the-art report covers the StyleGAN architecture, and the ways it has been employed since its conception, while also analyzing its severe limitations. It aims to be of use for both newcomers, who wish to get a grasp of the field, and for more experienced readers that might benefit from seeing current research trends and existing tools laid out. Among StyleGAN's most interesting aspects is its learned latent space. Despite being learned with no supervision, it is surprisingly well-behaved and remarkably disentangled. Combined with StyleGAN's visual quality, these properties gave rise to unparalleled editing capabilities. However, the control offered by StyleGAN is inherently limited to the generator's learned distribution, and can only be applied to images generated by StyleGAN itself. Seeking to bring StyleGAN's latent control to real-world scenarios, the study of GAN inversion and latent space embedding has quickly gained in popularity. Meanwhile, this same study has helped shed light on the inner workings and limitations of StyleGAN. We map out StyleGAN's impressive story through these investigations, and discuss the details that have made StyleGAN the go-to generator. We further elaborate on the visual priors StyleGAN constructs, and discuss their use in downstream discriminative tasks. Looking forward, we point out StyleGAN's limitations and speculate on current trends and promising directions for future research, such as task and target specific fine-tuning.

show abstract

Stitch it in Time: GAN-Based Facial Editing of Real Videos

Cited by 47 publications

References 31 publications

In-N-Out: Face Video Inversion and Editing with Volumetric Decomposition

In-N-Out: Face Video Inversion and Editing with Volumetric Decomposition

DisCO: Portrait Distortion Correction with Perspective-Aware 3D GANs

State‐of‐the‐Art in the Architecture, Methods and Applications of StyleGAN

Contact Info

Product

Resources

About