Recurrent Convolutional Structures for Audio Spoof and Video Deepfake Detection

Chintha, Akash; Thai, Bao; Sohrawardi, Saniat Javid; Bhatt, Kartavya; Hickerson, Andrea; Wright, Matthew; Ptucha, Raymond

doi:10.1109/jstsp.2020.2999185

Cited by 121 publications

(48 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We see that many approaches were proposed to apply frame-by-frame analysis in videos or images to manipulate face and track facial movement to obtain better performance. For example, in [66]- [71], RNN based networks are proposed to extract the features at various micro and macroscopic levels for detecting Deepfake. Regardless of these exciting results in detection, it is seen that most of the methods lean towards overfitting.…”

Section: ) Deep Learning Based Methodsmentioning

confidence: 99%

“…• The information is stored in a permission-based Blockchain, which gives the owner control over its contents. Based on the studies, taking together all these methods, Table 3 [11], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [45], [46], [47], [48], [49], [50], [51], [52], [53], [54], [55], [56], [57], [58], [59], [61], [62], [63], [64], [65], [66], [67], [68], [69], [70], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83], [84], [86],…”

Section: • Integrates the Critical Features Of Ipfs [114]-basedmentioning

confidence: 99%

See 1 more Smart Citation

Deepfake Detection: A Systematic Literature Review

et al. 2022

View full text Add to dashboard Cite

Over the last few decades, rapid progress in AI, machine learning, and deep learning has resulted in new techniques and various tools for manipulating multimedia. Though the technology has been mostly used in legitimate applications such as for entertainment and education, etc., malicious users have also exploited them for unlawful or nefarious purposes. For example, high-quality and realistic fake videos, images, or audios have been created to spread misinformation and propaganda, foment political discord and hate, or even harass and blackmail people. The manipulated, high-quality and realistic videos have become known recently as Deepfake. Various approaches have since been described in the literature to deal with the problems raised by Deepfake. To provide an updated overview of the research works in Deepfake detection, we conduct a systematic literature review (SLR) in this paper, summarizing 112 relevant articles from 2018 to 2020 that presented a variety of methodologies. We analyze them by grouping them into four different categories: deep learning-based techniques, classical machine learning-based methods, statistical techniques, and blockchain-based techniques. We also evaluate the performance of the detection capability of the various methods with respect to different datasets and conclude that the deep learning-based methods outperform other methods in Deepfake detection.

show abstract

Section: ) Deep Learning Based Methodsmentioning

confidence: 99%

Section: • Integrates the Critical Features Of Ipfs [114]-basedmentioning

confidence: 99%

Deepfake Detection: A Systematic Literature Review

et al. 2022

View full text Add to dashboard Cite

show abstract

“…In order to make full use of the structure information of the forged image, the local structure map is constructed and used as a weighting factor to highlight the structural loss in the forged image when counting the normalized histogram of LBP features, as shown in formula (7). The construction method of the local structure map MP is used to extract the phase congruency (PC) [44] features from the facial grey-scale image, and then select the larger one of the PC and the MSCN coefficient at each pixel point, as shown in formula (8).…”

Section: Texture Features Extraction Based On Structural Weightingmentioning

confidence: 99%

“…According to our findings, most of the existing detection methods [5–8] are mainly targeted at masking forgery, and a small part of researches is adapted to unmasking DeepFake [9–11], while few methods are suitable for both types at the same time. Thus, the application range of these single‐type forgery detection models is greatly restricted.…”

Section: Introductionmentioning

confidence: 99%

Deep forgery discriminator via image degradation analysis

Zhang

et al. 2021

IET Image Processing

View full text Add to dashboard Cite

Generative adversarial network-based deep generative model is widely applied in creating hyper-realistic face-swapping images and videos. However, its malicious use has posed a great threat to online contents, thus making detecting the authenticity of images and videos a tricky task. Most of the existing detection methods are only suitable for one type of forgery and only work for low-quality tampered images, restricting their applications. This paper concerns the construction of a novel discriminator with better comprehensive capabilities. Through analysis of the visual characteristics of manipulated images from the perspective of image quality, it is revealed that the synthesized face does have different degrees of quality degradation compared to the source content. Therefore, several kinds of image quality-related handicraft features are extracted, including texture, sharpness, frequency domain features, and deep features, to unveil the inconsistent information and modification traces in the fake faces. In this way, a 1065-dimensional vector of each image is obtained through multi-feature fusion, and it is then fed into RF to train a targeted binary classification detector. Extensive experiments have shown that the proposed scheme is superior to the previous methods in recognition accuracy on multiple manipulation databases including the Celeb-DF database with better visual quality.

show abstract

“…Dinkel et al proposed an end-to-end model with raw waveform input [9]. Chintha et al devised recurrent convolutional structures for audio spoofing detection [10]. In [11], a light convolutional neural network (LCNN) with angular margin based softmax loss was used for anti-spoofing attacks.…”

Section: Introductionmentioning

confidence: 99%

A Capsule Network Based Approach for Detection of Audio Spoofing Attacks

Luo

Liu

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Audio spoofing attacks not only increasingly pose a threat to automatic speaker verification systems but also have the potential to destabilize national security (e.g., by creating fake audio of influential politicians). The main purpose of anti-spoofing is to detect fake audios synthesized by advanced methods, while current algorithms using convolutional neural networks as classifiers exposed poor generalization to the unknown attacks. In this paper, as the first attempt, we introduce a capsule network to enhance the generalization of the detection system. To make the capsule network suitable for anti-spoofing tasks, we modified the original dynamic routing algorithm to force the model to pay more attention to artifacts and thus yield better detection performance for text-to-speech/voice conversion attacks. Furthermore, replay attack detection is also investigated, and the results indicate that our proposed approach is also highly capable of detecting replay attacks.

show abstract

Recurrent Convolutional Structures for Audio Spoof and Video Deepfake Detection

Cited by 121 publications

References 36 publications

Deepfake Detection: A Systematic Literature Review

Deepfake Detection: A Systematic Literature Review

Deep forgery discriminator via image degradation analysis

A Capsule Network Based Approach for Detection of Audio Spoofing Attacks

Contact Info

Product

Resources

About