DeepFaceLab: Integrated, flexible and extensible face-swapping framework

Perov, Ivan; Gao, Daiheng; Chervoniy, Nikolay; Liu, Kunlin; Marangonda, Sugasa; Umé, Chris; Dpfks,; Facenheim, Carl Shift; Rp, Luis; Jian, Jiang; Zhang, Sheng; Wu, Pingyu; Bo, Zhou; Zhang, Weimin

doi:10.48550/arxiv.2005.05535

Cited by 75 publications

(125 citation statements)

References 30 publications

Supporting

Mentioning

105

Contrasting

Order By: Relevance

“…FaceSwap: The FaceSwap videos are created using the person-specific impersonator videos by replacing the face of an impersonator with the target person's face. We obtain the FaceSwap videos for Obama, Biden, and Trump from WLDR and create the videos for Harris, Oliver and O'Brien using the open-source FaceSwap library [35]. The example frames are shown in the fourth column of Figure 3 and the duration of the videos (in hours) is given in Table 1.…”

Section: Impersonatormentioning

confidence: 99%

“…Our experiments include several world leaders and TV talk-show hosts and we consider the full spectrum of cutting edge deepfake and video manipulation techniques [35,36,41], as well as videos found in the wild. We compare our approach to several prominent prior works, and we show that we achieve the best AUC in all scenarios but one, performing well across the entire range of fakes.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion

Agarwal¹,

Hu²,

Ng³

et al. 2021

Preprint

View full text Add to dashboard Cite

In today's era of digital misinformation, we are increasingly faced with new threats posed by video falsification techniques. Such falsifications range from cheapfakes (e.g., lookalikes or audio dubbing) to deepfakes (e.g., sophisticated AI media synthesis methods), which are becoming perceptually indistinguishable from real videos. To tackle this challenge, we propose a multi-modal semantic forensic approach to discover clues that go beyond detecting discrepancies in visual quality, thereby handling both simpler cheapfakes and visually persuasive deepfakes. In this work, our goal is to verify that the purported person seen in the video is indeed themselves by detecting anomalous correspondences between their facial movements and the words they are saying. We leverage the idea of attribution to learn person-specific biometric patterns that distinguish a given speaker from others. We use interpretable Action Units (AUs) to capture a persons' face and head movement as opposed to deep CNN visual features, and we are the first to use word-conditioned facial motion analysis. Unlike existing person-specific approaches, our method is also effective against attacks that focus on lip manipulation. We further demonstrate our method's effectiveness on a range of fakes not seen in training including those without video manipulation, that were not addressed in prior work.1 A later report explained that in fact it was an impersonator [45].

show abstract

Section: Impersonatormentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion

Agarwal¹,

Hu²,

Ng³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…4, all actors agreed to use and modify their likeness. We use five methods: DeepFace-Lab [27,49], Faceswap [15,21,33], Faceswap-GAN [52], Recycle-GAN [2,3] and ALAE [50,51] to generate face In addition, we use source video frames s i ∈ S and fake video frames f i ∈ F to make mask video frames m i ∈ M, the masks define as…”

Section: Datasets and Setupmentioning

confidence: 99%

Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos

Pei¹,

Zhao²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Conventional fake video detection methods outputs a possibility value or a suspected mask of tampering images. However, such unexplainable results cannot be used as convincing evidence. So it is better to trace the sources of fake videos. The traditional hashing methods are used to retrieve semantic-similar images, which can't discriminate the nuances of the image. Specifically, the sources tracing compared with traditional video retrieval. It is a challenge to find the real one from similar source videos. We designed a novel loss Hash Triplet Loss to solve the problem that the videos of people are very similar: the same scene with different angles, similar scenes with the same person. We propose Vision Transformer based models named Video Tracing and Tampering Localization (VTL). In the first stage, we train the hash centers by ViTHash (VTL-T). Then, a fake video is inputted to ViTHash, which outputs a hash code. The hash code is used to retrieve the source video from hash centers. In the second stage, the source video and fake video are inputted to generator (VTL-L). Then, the suspect regions are masked to provide auxiliary information. Moreover, we constructed two datasets: DFTL and DAVIS2016-TL. Experiments on DFTL clearly show the superiority of our framework in sources tracing of similar videos. In particular, the VTL also achieved comparable performance with state-of-the-art methods on DAVIS2016-TL. Our source code and datasets have been released on GitHub: https://github.com/lajlksdf/vtl.

show abstract

“…Then, the resulting features in the latent space are input to the decoder trained on B. This principle is applied in several manipulation tools, like DeepFaceLab [12].…”

Section: A Synthesis Of Deepfake Videosmentioning

confidence: 99%

Detection of GAN-synthesized street videos

Alamayreh¹,

Barni²

2021

Preprint

View full text Add to dashboard Cite

Research on the detection of AI-generated videos has focused almost exclusively on face videos, usually referred to as deepfakes. Manipulations like face swapping, face reenactment and expression manipulation have been the subject of an intense research with the development of a number of efficient tools to distinguish artificial videos from genuine ones. Much less attention has been paid to the detection of artificial non-facial videos. Yet, new tools for the generation of such kind of videos are being developed at a fast pace and will soon reach the quality level of deepfake videos. The goal of this paper is to investigate the detectability of a new kind of AI-generated videos framing driving street sequences (here referred to as DeepStreets videos), which, by their nature, can not be analysed with the same tools used for facial deepfakes. Specifically, we present a simple frame-based detector, achieving very good performance on state-of-the-art DeepStreets videos generated by the Vid2vid architecture. Noticeably, the detector retains very good performance on compressed videos, even when the compression level used during training does not match that used for the test videos.

show abstract

DeepFaceLab: Integrated, flexible and extensible face-swapping framework

Cited by 75 publications

References 30 publications

Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion

Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion

Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos

Detection of GAN-synthesized street videos

Contact Info

Product

Resources

About