Deepfake forensics analysis: An explainable hierarchical ensemble of weakly supervised models

Silva, Samuel Henrique; Bethany, Mazal; Votto, Alexis Megan; Scarff, Ian; Beebe, Nicole; Najafirad, Peyman

doi:10.1016/j.fsisyn.2022.100217

Cited by 27 publications

(16 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The insights derived from this review aim to facilitate the development of novel and more robust approaches to address the growing complexities associated with deepfakes. Silva et al [1] introduces a hierarchical and explainable forensics algorithm that involves human input in the detection process. The data curation is carried out using a deep learning detection algorithm, and the decision is presented to humans along with a comprehensive set of forensic analyses on the decision region.…”

Section: Review Of Literature Much Research Work Has Been Conducted O...mentioning

confidence: 99%

“…These advancements in deepfake technology underscore the potential consequences of digitally manipulating facial features. While these applications offer entertainment and convenience, they also introduce a clear and significant risk, as malicious actors may exploit deepfake attacks to compromise the safety and security of individuals [1]. Over the past decade, there has been an exponential increase in the online presence of social media content, including photos and videos.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Untitled

2024

IJTRET

View full text Add to dashboard Cite

Deepfake technology, a fusion of deep learning and artificial intelligence, has emerged as a potent tool capable of crafting hyper-realistic yet entirely fabricated multimedia content. This comprehensive review explores the evolution, applications, and underlying principles of deepfake technology, emphasizing its potential implications for privacy, security, and the spread of misinformation. Using advanced deep learning algorithms, particularly Generative Adversarial Networks (GANs), deepfake technology manipulates facial features with remarkable precision, raising concerns about its malicious applications. The review examines the exponential growth in online content sharing facilitated by social media platforms and affordable devices, highlighting the convenience and accessibility but also the risks associated with the widespread use of deepfake technology. The core of deepfake technology, GANs, engages in an iterative competition between a discriminator and a generator, resulting in increasingly convincing synthetic data. A thorough review of the literature reveals significant research efforts focused on deepfake detection, leveraging techniques such as error-level analysis, CNN architectures, and hybrid approaches. The paper discusses regulatory measures, public awareness campaigns, and the critical role of digital forensic evaluation in mitigating deepfake threats. Challenges and concerns, including misinformation, privacy invasion, national security risks, and erosion of trust, are outlined. Mitigation strategies encompass advanced detection algorithms, regulatory frameworks, public awareness, and digital forensic evaluation, emphasizing the collaborative efforts required across technology developers, policymakers, the public, and digital forensic experts to navigate this evolving landscape and safeguard trust, privacy, and security in the digital age.

show abstract

Section: Review Of Literature Much Research Work Has Been Conducted O...mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Untitled

2024

IJTRET

View full text Add to dashboard Cite

show abstract

“…The accuracy rate of the proposed DFC model, a larger stacking ensemble neural network, was found to be 99.65%. Moreover, the ensemble hierarchical model proposed by Silva et al (2022) incorporated human involvement in the detection process by utilizing detection networks that employed both standard and attention-based data augmentation techniques. Attention blocks were utilized to assess facial regions, while human analysis of the frequency and statistical analyses of the region revealed by the explanation layer were used to ascertain the validity of the frame.…”

Section: Introductionmentioning

confidence: 99%

CNN-GMM approach to identifying data distribution shifts in forgeries caused by noise: a step towards resolving the deepfake problem

Alnafea,

Nissirat,

Al-Samawi

2024

PeerJ Computer Science

View full text Add to dashboard Cite

Recently, there have been notable advancements in video editing software. These advancements have allowed novices or those without access to advanced computer technology to generate videos that are visually indistinguishable to the human eye from real ones to the human observer. Therefore, the application of deepfake technology has the potential to expand the scope of identity theft, which poses a significant risk and a formidable challenge to global security. The development of an effective approach for detecting fake videos is necessary. Here, we introduce a novel methodology that employs a convolutional neural network (CNN) and Gaussian mixture model (GMM) to effectively differentiate between fake and real images or videos. The proposed methodology presents a novel CNN-GMM architecture in which the fully connected (FC) layer in the CNN is replaced with a customized Gaussian mixture model (GMM) fully connected layer. The GMM layer utilizes a weighted set of Gaussian probability density functions (PDFs) to represent the distribution of data frequencies in both real and fake images. This representation indicates there is a shift in the distribution of the manipulated images due to added noise. The CNN-GMM model demonstrates the ability to accurately identify variations resulting from different types of deepfakes within the probability distribution. It achieves a high level of classification accuracy, reaching up to 100% in training accuracy and up to 96% in validation accuracy. Notwithstanding the ratio of the genuine class to the counterfeit class being 16.6% to 83.4%, the CNN-GMM model exhibited high-performance metrics in terms of recall, accuracy, and F-score when classifying the least genuine class.

show abstract

“…https://www.indjst.org/ For signal processing enthusiasts, (6) offers an accessible overview addressing training methods, construction approaches, and current challenges in the realm of deep learning. The research (7) contributes a comprehensive review of deep learning techniques for video anomaly detection, categorizing state-of-the-art methods based on their ability to differentiate between normal and abnormal events, along with their underlying assumptions. In a related vein, (8) explores the growing interest in anomaly detection in video surveillance systems, driven by the demand for automated tools that can identify unusual events in video streams to enhance public safety.…”

Section: Introductionmentioning

confidence: 99%

ResNet50 DeepFake Detector: Unmasking Reality

Borade,

Jain,

Patel

et al. 2024

IJST

View full text Add to dashboard Cite

Objectives: The objective of this research is to detect video deepfakes with a higher accuracy and provide optimum results. The research aims to reduce time complexity for the media processing while simultaneously working on the model accuracy. Methods: This research has utilized CelebDF and FaceForensics++ Datasets for training and 32 epochs with the use of Single Nvidia Tesla T4 GPU. The above method of training and validating the model yielded error of <5% and is very capable. Using image scraping this model initially eliminates the unimportant areas of consideration. Thus, reducing the amount of scans that the model does in order to identify the face. This reduces that training to graph ratio and provides less error margin. Findings: This research’s findings reveal the model's robustness in detecting manipulated videos generated by deepfake techniques. Through extensive experimentation on diverse datasets, ResNet50 consistently demonstrated 97% accuracy, sensitivity, and specificity in distinguishing authentic content from deepfakes. The model exhibited exceptional generalization across various scenarios, including face-swapping and lip-syncing, showcasing its adaptability to evolving deepfake techniques. This research contributes to the already existing literature on ResNet50 deepfake detection tools. It contributes by adding the image scraping feature to the ResNet50 model and overcomes the gaps such as increasing error percentage of some of the models. The research of 1 has a 20% error percentage while this research has an error percentage of 5% with an accuracy of 97%. Novelty: The study employs ResNet50 to detect deepfake videos, utilizing novel image scraping techniques to minimize errors and enhance prediction accuracy. Keywords: Deepfakes, Deep Learning, GAN, ResNet50, FaceForensics++, CelebDF

show abstract

Deepfake forensics analysis: An explainable hierarchical ensemble of weakly supervised models

Cited by 27 publications

References 9 publications

Untitled

Untitled

CNN-GMM approach to identifying data distribution shifts in forgeries caused by noise: a step towards resolving the deepfake problem

ResNet50 DeepFake Detector: Unmasking Reality

Contact Info

Product

Resources

About