Improving the Generalization Ability of Deepfake Detection via Disentangled Representation Learning

Hu, Jiashang; Wang, Shilin; Li, Xiaoyong

doi:10.1109/icip42928.2021.9506730

Cited by 7 publications

(3 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired by (Hsu et al, 2021), we introduce Lemma 11 to tackle the error probability Pr (x,y)∼µ arg max Then we will study the upper bound for E x,y ψ(g(x)) y in Lemma 8.…”

Section: B3 Proofs For Generalization Bounds Of Global Model Theoremmentioning

confidence: 99%

PerAda: Parameter-Efficient and Generalizable Federated Learning Personalization with Guarantees

Xie¹,

Huang²,

Wenda³

et al. 2023

Preprint

View full text Add to dashboard Cite

Personalized Federated Learning (pFL) has emerged as a promising solution to tackle data heterogeneity across clients in FL. However, existing pFL methods either (1) introduce high communication and computation costs or (2) overfit to local data, which can be limited in scope, and are vulnerable to evolved test samples with natural shifts. In this paper, we propose PERADA, a parameter-efficient pFL framework that reduces communication and computational costs and exhibits superior generalization performance, especially under test-time distribution shifts. PERADA reduces the costs by leveraging the power of pretrained models and only updates and communicates a small number of additional parameters from adapters. PERADA has good generalization since it regularizes each client's personalized adapter with a global adapter, while the global adapter uses knowledge distillation to aggregate generalized information from all clients. Theoretically, we provide generalization bounds to explain why PERADA improves generalization, and we prove its convergence to stationary points under non-convex settings. Empirically, PERADA demonstrates competitive personalized performance (+4.85% on CheXpert) and enables better out-of-distribution generalization (+5.23% on CIFAR-10-C) on different datasets across natural and medical domains compared with baselines, while only updating 12.6% of parameters per model based on the adapter.

show abstract

“…Inspired by (Hsu et al, 2021), we introduce Lemma 11 to tackle the error probability Pr (x,y)∼µ arg max Then we will study the upper bound for E x,y ψ(g(x)) y in Lemma 8.…”

Section: B3 Proofs For Generalization Bounds Of Global Model Theoremmentioning

confidence: 99%

PerAda: Parameter-Efficient and Generalizable Federated Learning Personalization with Guarantees

Xie¹,

Huang²,

Wenda³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…erefore, the conventional feature forensics technique cannot be directly applied to detect deepfake videos. Methods based on or combined with deep learning have recently gained attention [26][27][28][29]. Sabir et al [30] used recurrent neural networks to capture temporal differences in fake videos.…”

Section: Related Workmentioning

confidence: 99%

Multisemantic Path Neural Network for Deepfake Detection

Jin

Jiang

et al. 2022

Security and Communication Networks

View full text Add to dashboard Cite

With the continuous development of deep learning techniques, it is now easy for anyone to swap faces in videos. Researchers find that the abuse of these techniques threatens cyberspace security; thus, face forgery detection is a popular research topic. However, current detection methods do not fully use the semantic features of deepfake videos. Most previous work has only divided the semantic features, the importance of which may be unequal, by experimental experience. To solve this problem, we propose a new framework, which is the multisemantic pathway network (MSPNN) for fake face detection. This method comprehensively captures forged information from the dimensions of microscopic, mesoscopic, and macroscopic features. These three kinds of semantic information are given learnable weights. The artifacts of deepfake images are more difficult to observe in a compressed video. Therefore, preprocessing is proposed to detect low-quality deepfake videos, including multiscale detail enhancement and channel information screening based on the compression principle. Center loss and cross-entropy loss are combined to further reduce intraclass spacing. Experimental results show that MSPNN is superior to contrast methods, especially low-quality deepfake video detection.

show abstract

“…See also the follow up works (Harutyunyan et al, 2021;Haghifam et al, 2021) and (Steinke and Zakynthinou, 2020). The roots of compression-based approaches perhaps date back to Littlestone and Warmuth (1986) who studied the predictability of the training data labels using only part of the dataset; and this has then been extended and used in various ways, see, e.g., (Arora et al, 2018;Suzuki et al, 2020;Hsu et al, 2021;Barsbey et al, 2021) and the recent (Sefidgaran et al, 2022). The fractal-based approach is a recently initiated line of work that hinges on that when the algorithm has a recursive nature, e.g., it involves an iterative optimization procedure, it might generate a fractal structure either in the model trajectories (S ¸ims ¸ekli et al, 2020;Birdal et al, 2021;Hodgkinson et al, 2022;Lim et al, 2022) or in its distribution (Camuto et al, 2021).…”

Section: Introduction and Problem Setupmentioning

confidence: 99%

Data-dependent Generalization Bounds via Variable-Size Compressibility

Sefidgaran¹,

Zaidi²

2023

Preprint

View full text Add to dashboard Cite

In this paper, we establish novel data-dependent upper bounds on the generalization error through the lens of a "variable-size compressibility" framework that we introduce newly here. In this framework, the generalization error of an algorithm is linked to a variable-size 'compression rate' of its input data. This is shown to yield bounds that depend on the empirical measure of the given input data at hand, rather than its unknown distribution. Our new generalization bounds that we establish are tail bounds, tail bounds on the expectation, and in-expectations bounds. Moreover, it is shown that our framework also allows to derive general bounds on any function of the input data and output hypothesis random variables. In particular, these general bounds are shown to subsume and possibly improve over several existing PAC-Bayes and data-dependent intrinsic dimension-based bounds that are recovered as special cases, thus unveiling a unifying character of our approach. For instance, a new data-dependent intrinsic dimension based bounds is established, which connects the generalization error to the optimization trajectories and reveals various interesting connections with rate-distortion dimension of process, Rényi information dimension of process, and metric mean dimension.

show abstract

Improving the Generalization Ability of Deepfake Detection via Disentangled Representation Learning

Cited by 7 publications

References 22 publications

PerAda: Parameter-Efficient and Generalizable Federated Learning Personalization with Guarantees

PerAda: Parameter-Efficient and Generalizable Federated Learning Personalization with Guarantees

Multisemantic Path Neural Network for Deepfake Detection

Data-dependent Generalization Bounds via Variable-Size Compressibility

Contact Info

Product

Resources

About