Detecting and Recovering Sequential DeepFake Manipulation

Shao, Rui; Wu, Tianxing; Liu, Ziwei

doi:10.1007/978-3-031-19778-9_41

Cited by 16 publications

(2 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These techniques often rely on discrepancies between the body/ head of the underlying person, and the deepfake face that is superimposed onto the original video. These include quality differences, where the inner face appears blurred or pixelated compared to the surrounding video (Younus & Hasan, 2020), differences in skin tone or texture between the original and new face (Ajoy et al, 2021), and inconsistent blending at the contours of the face (Shao et al, 2022). Similar artifacts are reported by humans who correctly identify deepfakes (Wöhler et al, 2021).…”

Section: Puppetry Deepfakes For Emotion Perception Researchmentioning

confidence: 76%

Can deepfakes be used to study emotion perception? A comparison of dynamic face stimuli

Becker,

Conduit,

Chouinard

et al. 2024

Behav Res

View full text Add to dashboard Cite

Video recordings accurately capture facial expression movements; however, they are difficult for face perception researchers to standardise and manipulate. For this reason, dynamic morphs of photographs are often used, despite their lack of naturalistic facial motion. This study aimed to investigate how humans perceive emotions from faces using real videos and two different approaches to artificially generating dynamic expressions – dynamic morphs, and AI-synthesised deepfakes. Our participants perceived dynamic morphed expressions as less intense when compared with videos (all emotions) and deepfakes (fearful, happy, sad). Videos and deepfakes were perceived similarly. Additionally, they perceived morphed happiness and sadness, but not morphed anger or fear, as less genuine than other formats. Our findings support previous research indicating that social responses to morphed emotions are not representative of those to video recordings. The findings also suggest that deepfakes may offer a more suitable standardized stimulus type compared to morphs. Additionally, qualitative data were collected from participants and analysed using ChatGPT, a large language model. ChatGPT successfully identified themes in the data consistent with those identified by an independent human researcher. According to this analysis, our participants perceived dynamic morphs as less natural compared with videos and deepfakes. That participants perceived deepfakes and videos similarly suggests that deepfakes effectively replicate natural facial movements, making them a promising alternative for face perception research. The study contributes to the growing body of research exploring the usefulness of generative artificial intelligence for advancing the study of human perception.

show abstract

Section: Puppetry Deepfakes For Emotion Perception Researchmentioning

confidence: 76%

Can deepfakes be used to study emotion perception? A comparison of dynamic face stimuli

Becker,

Conduit,

Chouinard

et al. 2024

Behav Res

View full text Add to dashboard Cite

show abstract

“…Shao et al [20] proposed the Seq-DeepFake Transformer (SeqFakeFormer) to detect forged images. First, they captured spatial manipulation traces of the image through self-attention modules in the transformer encoder, and then added the Spatially Enhanced Cross-Attention (SECA) module to generate different spatial weight maps for corresponding manipulations to carry out cross-attention.…”

Section: Detection Based On Video Intraframe Featuresmentioning

confidence: 99%

Deepfake Video Detection Based on Improved CapsNet and Temporal–Spatial Features

Lu¹,

Bao²,

Li³

2023

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

Rapid development of deepfake technology led to the spread of forged audios and videos across network platforms, presenting risks for numerous countries, societies, and individuals, and posing a serious threat to cyberspace security. To address the problem of insufficient extraction of spatial features and the fact that temporal features are not considered in the deepfake video detection, we propose a detection method based on improved CapsNet and temporal-spatial features (iCapsNet-TSF). First, the dynamic routing algorithm of CapsNet is improved using weight initialization and updating. Then, the optical flow algorithm is used to extract interframe temporal features of the videos to form a dataset of temporal-spatial features. Finally, the iCapsNet model is employed to fully learn the temporal-spatial features of facial videos, and the results are fused. Experimental results show that the detection accuracy of iCapsNet-TSF reaches 94.07%, 98.83%, and 98.50% on the Celeb-DF, FaceSwap, and Deepfakes datasets, respectively, displaying a better performance than most existing mainstream algorithms. The iCapsNet-TSF method combines the capsule network and the optical flow algorithm, providing a novel strategy for the deepfake detection, which is of great significance to the prevention of deepfake attacks and the preservation of cyberspace security.

show abstract