DFT-MF: Enhanced deepfake detection using mouth movement and transfer learning

Elhassan, Ammar; Al-Fawa’reh, Mohammad; Jafar, Mousa Tayseer; Ababneh, Mohammad; Jafar, Shifaa  Tayseer

doi:10.1016/j.softx.2022.101115

Cited by 15 publications

(9 citation statements)

References 22 publications

(1 reference statement)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consequently, the proposed model was trained on all parts of the face separately and with the same hyperparameters shown in Section (5) and the same images from the same database explained in Section 3. The results later demonstrate that all parts are valid to be alone sufficient in identifying and detecting deep forgery and provided good and better results than some previous studies.…”

Section: Comparative Evaluation Of the Face Segments Resultsmentioning

confidence: 99%

“…Note that there are numerous methods proposed to detect Deepfake, some of which rely on the use of pretrained models like Xception, Inception, EfficientNet, etc., because of their ability to extract manipulation features. Others started building their own convolutional neural network (CNN), such as [5], [6], etc., because they were interested in working on some of the specific features in the image. In this paper, the pre-trained model DenseNet121 [7] is used as the input layer to extract features map from images, followed by a few layers to detect whether it is fake or not.…”

Section: Introductionmentioning

confidence: 99%

“…This will make the model noisier, hurting how well it works. Another pre-processing that is based only on the eyes [11], mouth, or teeth [5] as a feature will also have many limitations, like the face being in front and the eyes being open or the mouth being open and the teeth being visible and so on. Thus, it is necessary to intensify efforts to demonstrate the importance and effectiveness of pre-processing by avoiding these limitations to increase the network results.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deploying Facial Segmentation Landmarks for Deepfake Detection

Abdullah,

M. Ali

2023

J. Al-Qadisiyah Comp. Sci. Math.

View full text Add to dashboard Cite

Deepfake is a type of artificial intelligence used to create convincing images, audio, and video hoaxes and it concerns celebrities and everyone because they are easy to manufacture. Deepfake are hard to recognize by people and current approaches, especially high-quality ones. As a defense against Deepfake techniques, various methods to detect Deepfake in images have been suggested. Most of them had limitations, like only working with one face in an image. The face has to be facing forward, with both eyes and the mouth open, depending on what part of the face they worked on. Other than that, a few focus on the impact of pre-processing steps on the detection accuracy of the models. This paper introduces a framework design focused on this aspect of the Deepfake detection task and proposes pre-processing steps to improve accuracy and close the gap between training and validation results with simple operations. Additionally, it differed from others by dealing with the positions of the face in various directions within the image, distinguishing the concerned face in an image containing multiple faces, and segmentation the face using facial landmarks points. All these were done using face detection, face box attributes, facial landmarks, and key points from the MediaPipe tool with the pre-trained model (DenseNet121). Lastly, the proposed model was evaluated using Deepfake Detection Challenge datasets, and after training for a few epochs, it achieved an accuracy of 97% in detecting the Deepfake

show abstract

Section: Comparative Evaluation Of the Face Segments Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deploying Facial Segmentation Landmarks for Deepfake Detection

Abdullah,

M. Ali

2023

J. Al-Qadisiyah Comp. Sci. Math.

View full text Add to dashboard Cite

show abstract

“…Using 12 aural/oral associations between the real videos and the simulated false ones, a multinomial logit model is developed. Then, utilizing real movies together with simulation, Generative adversarial network, and in-the-wild false films, this model was assessed [17].…”

Section: Ear and Mouth Movementmentioning

confidence: 99%

Deepfake Detection using Biological Features: A Survey

Patil¹,

Kale²,

Dhokey³

et al. 2023

Preprint

View full text Add to dashboard Cite

A deep learning-based technique called deepfake has made it easier to change or modify images and videos. In investigations and court, visual evidence is commonly employed. These pieces of evidence may now be a suspect due to technological advancements, particularly Deepfake. Photographs and movies that have been edited are incredibly lifelike and difficult to tell apart from the original. Deepfakes have been utilized to blackmail individuals, plan terrorist attacks, disseminate incorrect facts, defame individuals as well as foment political turmoil. Our study describes the history of the Deepfake to acquire a thorough understanding of the technology. Additionally, the study focuses on the development and detection of Deepfakes and their challenges based on physiological measurements. Here, biological features such as eyebrow recognition, eye blinking detection, eye movement detection, ear and mouth detection, and heartbeat detection are clearly described and a scope in this domain is proposed. A comparison of each biological feature concerning classifiers or techniques along with its key findings is discussed. Generated using the generative adversarial network (GANs) model, Deepfakes are created by iterating an actual data-based generation and verification task through two opposite deep learning models. It was easier to identify deepfakes by humans during the nascent stage of this technology owing to pixel collapse phenomena that generated visible artifacts in skin tone and the general face structure. However, with the technology's advancement, DeepFakes have evolved to be highly indistinguishable from natural images hence it is important to review these detection methods.

show abstract

“…The performance of detection model is enhanced by filtering out the low-frequency information from the facial image while retaining the high-frequency information with high texture discrimination through the use of a novel pre-processing technique. Elhassan, A., et al [10] introduced a deep learning method that utilizes mouth and teeth movements to detect fake videos. as a differentiating feature that are still very challenging to handle when making fake videos.…”

mentioning

confidence: 99%

DeepFake Video Detection using Vision Transformer

Hussien,

Mohamed

2024

IJICIS

View full text Add to dashboard Cite

Technology is always a double-edged sword, and with the astonishing advancements in technology, it is expected that the DeepFake problem will become more common and serious. DeepFake has recently caused a lot of trouble because its flaws outweigh its advantages. Since DeepFake has such a significant influence on individuals deception, instability of principles and falsification of evidence. Instead of just affecting people, it led to multiple incidents that affected the image of entire nations. In this paper, a model that has been built to mitigate the negative effects of deepFake and maintain an individual's reputation by detecting the alteration of people's photographs and videos. A model with integrated vision transformer architectures Deep-ViT and Cross-ViT is designed to process pre-extracted faces from FF++ dataset. The model distinguishes between the real and fake faces in two different perspectives, subclass detection on each manipulation method and overall detection of all types. The proposed model achieves an outstanding results and the highest accuracy in FaceSwap manipulation method with 98%.

show abstract

DFT-MF: Enhanced deepfake detection using mouth movement and transfer learning

Cited by 15 publications

References 22 publications

Deploying Facial Segmentation Landmarks for Deepfake Detection

Deploying Facial Segmentation Landmarks for Deepfake Detection

Deepfake Detection using Biological Features: A Survey

DeepFake Video Detection using Vision Transformer

Contact Info

Product

Resources

About