DeepFake Videos Detection Using Self-Supervised Decoupling Network

Zhang, Jian; Ni, Jiangqun; Xie, Hao

doi:10.1109/icme51207.2021.9428368

Cited by 17 publications

(5 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, identifying biological anomalies (such as teeth, forehead, heart rate, etc. ), the correlation between the visual and audio of faces in the video [27][28][29][30], and detecting inter-frame anomaly of fake video [31][32][33][34][35][36][37]. Furthermore, unique features are employed to detect deepfakes on POI.…”

Section: Related Workmentioning

confidence: 99%

Spatial-temporal transformer network for protecting person-of-interest from deepfaking

Zhang

et al. 2023

Preprint

View full text Add to dashboard Cite

The rampant use of forgery techniques poses a significant threat to the security of celebrities' identities. Although current deepfake detection methods have shown effectiveness when dealing with specific public face forgery datasets, their reliability diminishes when applied to open data. Moreover, these methods are susceptible to re-compression and mainly rely on pixel-level abnormalities in forgery faces. In this study, we present a novel approach to detecting face forgery by leveraging individual speaking patterns of facial expressions and head movements. Our method utilizes potential motion patterns and inter-frame variations to effectively differentiate between fake and real videos. We propose an end-to-end dual-branch detection network, named the spatial-temporal transformer (STT), which aims to safeguard the identity of the person-of-interest (POI) from deepfaking. The STT incorporates the spatial transformer (ST) to establish the connection between facial expressions and head movements, while the temporal transformer (TT) exploits inconsistencies in facial attribute changes. Additionally, we introduce a central compression loss to enhance the detection performance. Extensive experiments are conducted to evaluate the effectiveness of the STT, and the results demonstrate its superiority over other SOTA methods in detecting forgery videos involving POIs. Furthermore, our network exhibits resilience to pixel-level re-compression perturbations, making it a robust solution in the face of evolving forgery techniques.

show abstract

Section: Related Workmentioning

confidence: 99%

Spatial-temporal transformer network for protecting person-of-interest from deepfaking

Zhang

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…However, the capsule network had lower detection rates with previously unseen deepfake videos. In [161], a self-supervised decoupling network (SDNN) for learning authenticity and compression features is proposed. As self-supervised signals, it used the compression ratio of given input images.…”

Section: Generic Neural Network Approachmentioning

confidence: 99%

DeepFake on Face and Expression Swap: A Review

Waseem,

Abu Bakar,

Ahmed

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Remarkable advances have been made in deep learning, leading to the emergence of highly realistic AIgenerated videos known as deepfakes. Deepfakes use generative models to manipulate facial features to create modified identities or expressions with impressive realism. These synthetic media creations can deceive, discredit, or blackmail individuals and threaten the integrity of the legal, political, and social systems. Consequently, researchers are actively developing techniques to detect deepfake content to preserve privacy and combat the dissemination of fabricated media. This article presents a comprehensive study examining existing methods of creating deepfake images and videos for face and expression replacement. In addition, it provides an overview of publicly available deepfake datasets for benchmarking, serving as important resources for training and evaluating deepfake detection systems. In addition, the study sheds light on the detection approaches used to identify deepfake face and expression swaps while discussing the challenges and issues involved. However, the focus of this study goes beyond identifying the existing barriers. It goes a step further by outlining future research directions and guiding future scientists to address the concerns that need to be addressed in deepfake detection methods. In this way, this paper aims to facilitate the development of robust and effective deepfake detection solutions for face and expression swaps, thereby contributing to ongoing efforts to protect the authenticity and trustworthiness of visual media.

show abstract

“…There are also many methods devoted to the identification of compressed fake data. Zhang et al 37 propose a self‐supervised decoupling network SSDN that incorporates compression independence. This method achieves good results on compressed fake datasets.…”

Section: Related Workmentioning

confidence: 99%

A novel forgery classification method based on multi‐scale feature capsule network in mobile edge computing

Lian

Wang

2023

Softw Pract Exp

View full text Add to dashboard Cite

Face recognition is one of the most important applications of MEC. However, there have been many fake face data that can deceive MEC devices, causing serious problems such as information leakage. Face forgery detection can effectively solve this problem. Current face forgery detection methods have achieved high accuracy. However, most of the methods are researched on the classification of face authenticity. We find that studying multi‐classification of forgery methods can not only improve the accuracy of the model to identify fake faces, but also help improve the generalization ability of fake face classification. We argue that multi‐scale features and high‐frequency features can expose more detailed forgery artifacts. So, we design four modules, which take advantage of the complementarity of RGB features and frequency features, global features and local features. The first module is a residual‐guided multi‐scale spatial attention module, which uses residuals to guide the RGB feature extractor to extract fake features from a multi‐scale perspective. The second module is a multi‐scale retinal feature extraction module. The third module is a multi‐scale channel attention‐guided local frequency statistics module, which extracts local frequency responses using sliding‐window DCT. The last module is a capsule network classification module with overall correlation to classify the fused features. The information transfer between the subject capsule and the classification capsule can increase the integrity of the model, making the model converge faster. We conduct experiments on the databases FaceForensics++, DeepfakeDetection, and FakeAVCeleb. Experimental result shows that our method performs well on forgery classification.

show abstract

DeepFake Videos Detection Using Self-Supervised Decoupling Network

Cited by 17 publications

References 21 publications

Spatial-temporal transformer network for protecting person-of-interest from deepfaking

Spatial-temporal transformer network for protecting person-of-interest from deepfaking

DeepFake on Face and Expression Swap: A Review

A novel forgery classification method based on multi‐scale feature capsule network in mobile edge computing

Contact Info

Product

Resources

About