High-Fidelity Face Swapping with Style Blending

Abstract

Face swapping is gaining significant traction, boosted by the plethora of human face synthesis with the deep learning methods. Recent works based on Generative Adversarial Nets (GAN) for face swapping often suffer from blending inconsistency, distortions and artefacts, as well as instability in training. In this work, we propose a novel end-to-end framework for high-fidelity face swapping, leveraging the high photorealistic face generation techniques from StyleGAN. Firstly, we invert the facial images into the style latent space by purposing a novel facial attributes encoder that is capable of extracting face essentials from the face image and projecting them to the style code in the latent space. We show that such inverted style code encapsulates facial attributes that are indispensable for face swapping task. Secondly, a carefully designed style blending module (SBM) is introduced for transferring the identity from a source image to the target by the multi-head attention (MHA) mechanism. We propose relevant constraints for guiding the learning of the SBM, leading to the effective blending of the Face ID from the source face to the target image. Finally, the blended style code can be translated back to the image space via the style decoder, benefiting from the training stability and the high quality of the generative capability of the style-based decoder. Extensive experiments demonstrate the superior quality of the face synthesis results (illustrated in Figure 1) of our face-swapping system compared with other state-of-the-art methods.

From Land to Ocean: Bathymetric Terrain Reconstruction via Conditional Generative Adversarial Network

“…The traditional interpolation methods can achieve good reconstruction results in areas of moderate size and relatively flat low-lying terrain, but their performance is poor in areas with large terrain fluctuations. In recent years, with the development of deep learning (DL), the DL methods have been applied in various fields (Yang et al, 2023a), including terrain reconstruction, image restoration, and super-resolution (Yang et al, 2023b;He et al, 2023;Alzahem et al, 2023). Generative adversarial network (GAN) (Goodfellow et al, 2014) is powerful generative models that learn deep features through adversarial training between the generator and discriminator.…”