Video face forgery detection via facial motion-assisted capturing dense optical flow truncation

Yang, Gaoming; Xu, Kun; Zhang, Ji

doi:10.1007/s00371-022-02683-z

Cited by 9 publications

(5 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They judge real and fake images by different degrees of similarity. Yang et al [21] found the dense optical flow of real and fake video does not behave consistently. They combined this phenomenon with the feature extraction network for video forgery detection.…”

Section: Detection Methods In the Spatial Domainmentioning

confidence: 99%

“…According to the form of data processed, we divide the current DeepFake detection methods into two categories, i.e., in the spatial domain [11,[18][19][20][21] and the frequency domain [14] .…”

Section: Related Workmentioning

confidence: 99%

“…In this section, we compare FDS_2D with some excellent detection methods that performed well in 2017-2022, like Twostream [35] and DTFA-DOF [21] , etc. The datasets used in the experiment include FF++_c23, FF++_c40, Celeb-DF, and DFDC.…”

Section: Comparisonmentioning

confidence: 99%

See 2 more Smart Citations

FDS_2D: Rethinking magnitude-phase features for DeepFake Detection

Yang

Wei

Zhang

2023

Preprint

Self Cite

View full text Add to dashboard Cite

The detection method based on the frequency domain is one of the essential methods to reduce the harm of forged information. This method mostly uses spectra as clues to identify fake content. However, the current methods tend to use only one of the magnitude and phase spectra for learning. In this paper, we notice that the magnitude and phase spectrum contain different image information. Only one spectrum is easily disturbed by noise, and the robustness of the method is difficult to guarantee. Therefore, we propose the Frequency Domain Separable DeepFake Detection (FDS_2D) to use a multi-branch network to obtain features in different frequency spectra. In FDS_2D, the spectral information is divided into three categories: the magnitude spectrum, the phase spectrum, and the relationship between the two spectra. According to their characteristics, we design independent methods for feature extraction from them. Moreover, to improve the utilization efficiency of multi-features, we propose a multi-input multi-output attention mechanism for information interaction between branches. The experimental results show that each part of FDS_2D effectively extracts and applies spectral information; The comprehensive performance of our model is verified on FaceForensic++, CelebDF, and DFDC. It proves that the ability of FDS_2D to detect DeepFake is not inferior to existing models.

show abstract

Section: Detection Methods In the Spatial Domainmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

FDS_2D: Rethinking magnitude-phase features for DeepFake Detection

Yang

Wei

Zhang

2023

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [24], authors introduced ViXNet, a two-branch network, one that attempts to learn discrepancies between local face area details by combining a patch-wise self-attention module with a ViT and the other uses XceptionNet to extract global spatial features. Yang et al [25] proposed a dense optical ow-based approach to detect forged faces. The authors revealed that forged video frames exhibit optical ow truncation by examining successive video frames processed using the optical ow technique.…”

Section: Video Deepfake Detectionmentioning

confidence: 99%

“…Multiple attempts have been made by researchers and forensics experts to develop several benchmark datasets [8][9][10][11][12] and algorithms for deepfake video detection [13][14][15][16][17][18][19][20][21][22][23][24][25][26]. Existing low-level approaches concentrate on inconsistent behaviors, including eye blinking [14], facial expression, head movement [15,16], lip movement [27,28], gestural mannerisms [29], and misalignment between the eyes [21].…”

Section: Introductionmentioning

confidence: 99%

Attention-based Multimodal learning framework for Generalized Audio- Visual Deepfake Detection

Masood,

Javed,

Irtaza

2023

Preprint

View full text Add to dashboard Cite

Deepfake media proliferated on the internet has major societal consequences for politicians, celebrities, and even common people. Recent advancements in deepfake videos include the creation of realistic talking faces and the usage of synthetic human voices. Numerous deepfake detection approaches have been proposed in response to the potential harm caused by deepfakes. However, the majority of deepfake detection methods process audio and video modality independently and have low identification accuracy. In this work, we propose an ensemble multimodal deepfake detection method that can identify both auditory and facial manipulations by exploiting correspondence between audio-visual modalities. The proposed framework comprises unimodal and cross-modal learning networks to exploit intra- and inter-modality inconsistencies introduced as a result of manipulation. The suggested multimodal approach employs an ensemble of deep convolutional neural-network based on an attention mechanism that extracts representative features and effectively determines if a video is fake or real. We evaluated the proposed approach on several benchmark multimodal deepfake datasets including FakeAVCeleb, DFDC-p, and DF-TIMIT. Experimental results demonstrate that an ensemble of deep learners based on unimodal and cross-modal network mechanisms exploit highly semantic information between audio and visual signals and outperforms independently trained audio and visual classifiers. Moreover, it can effectively identify different unseen types of deepfakes as well as robust under various post-processing attacks. The results confirm that our approach outperforms existing unimodal/multimodal classifiers for audio-visual manipulated video identification.

show abstract