A Comprehensive Review of Deep-Learning-Based Methods for Image Forensics

Camacho, Ivan Castillo; Wang, Kai

doi:10.3390/jimaging7040069

Cited by 48 publications

(22 citation statements)

References 148 publications

(247 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning based methods have been widely used, and are considered state-of-the-art cutting edge in what image and video forensics are about [ 35 , 40 ]. However, the features extraction methods and the overall functioning of deep learning based models, such as CNN and RNN, are time-consuming to process, and less flexible to be embedded into standalone off-the-shelf digital forensics tools, like Autopsy.…”

Section: Results Analysismentioning

confidence: 99%

“…Castillo and Yang [ 35 ] present a comprehensive review of the state-of-the-art deep learning-based methods for image forensics, both photos and videos. An exhaustive set of methods are introduced for a set of problems, namely median filtering, double JPEG, contrast enhancement, and general-purpose image processing operations.…”

Section: Literature Review and State Of The Artmentioning

confidence: 99%

See 1 more Smart Citation

Exposing Manipulated Photos and Videos in Digital Forensics Analysis

2021

View full text Add to dashboard Cite

Tampered multimedia content is being increasingly used in a broad range of cybercrime activities. The spread of fake news, misinformation, digital kidnapping, and ransomware-related crimes are amongst the most recurrent crimes in which manipulated digital photos and videos are the perpetrating and disseminating medium. Criminal investigation has been challenged in applying machine learning techniques to automatically distinguish between fake and genuine seized photos and videos. Despite the pertinent need for manual validation, easy-to-use platforms for digital forensics are essential to automate and facilitate the detection of tampered content and to help criminal investigators with their work. This paper presents a machine learning Support Vector Machines (SVM) based method to distinguish between genuine and fake multimedia files, namely digital photos and videos, which may indicate the presence of deepfake content. The method was implemented in Python and integrated as new modules in the widely used digital forensics application Autopsy. The implemented approach extracts a set of simple features resulting from the application of a Discrete Fourier Transform (DFT) to digital photos and video frames. The model was evaluated with a large dataset of classified multimedia files containing both legitimate and fake photos and frames extracted from videos. Regarding deepfake detection in videos, the Celeb-DFv1 dataset was used, featuring 590 original videos collected from YouTube, and covering different subjects. The results obtained with the 5-fold cross-validation outperformed those SVM-based methods documented in the literature, by achieving an average F1-score of 99.53%, 79.55%, and 89.10%, respectively for photos, videos, and a mixture of both types of content. A benchmark with state-of-the-art methods was also done, by comparing the proposed SVM method with deep learning approaches, namely Convolutional Neural Networks (CNN). Despite CNN having outperformed the proposed DFT-SVM compound method, the competitiveness of the results attained by DFT-SVM and the substantially reduced processing time make it appropriate to be implemented and embedded into Autopsy modules, by predicting the level of fakeness calculated for each analyzed multimedia file.

show abstract

Section: Results Analysismentioning

confidence: 99%

Section: Literature Review and State Of The Artmentioning

confidence: 99%

Exposing Manipulated Photos and Videos in Digital Forensics Analysis

2021

View full text Add to dashboard Cite

show abstract

“…Several surveys on the use of deep learning methods for digital forensics have been published recently [9,10]. The results obtained with CNN on image forensics are impressive and outperform those obtained with other machine learning methods.…”

Section: Background and Summarymentioning

confidence: 98%

“…Additional research should be made to reduce the processing time on using CNN in standalone digital forensics tools. [9,10]. Notwithstanding, the features extraction methods and the overall functioning of deep learning based models, such as CNN and RNN, are time-consuming to process and less flexible to be embedded into a standalone digital forensics application, such as Autopsy.…”

Section: Technical Validationmentioning

confidence: 99%

A Dataset of Photos and Videos for Digital Forensics Analysis Using Machine Learning Processing

Ferreira

Antunes

Correia

2021

Data

View full text Add to dashboard Cite

Deepfake and manipulated digital photos and videos are being increasingly used in a myriad of cybercrimes. Ransomware, the dissemination of fake news, and digital kidnapping-related crimes are the most recurrent, in which tampered multimedia content has been the primordial disseminating vehicle. Digital forensic analysis tools are being widely used by criminal investigations to automate the identification of digital evidence in seized electronic equipment. The number of files to be processed and the complexity of the crimes under analysis have highlighted the need to employ efficient digital forensics techniques grounded on state-of-the-art technologies. Machine Learning (ML) researchers have been challenged to apply techniques and methods to improve the automatic detection of manipulated multimedia content. However, the implementation of such methods have not yet been massively incorporated into digital forensic tools, mostly due to the lack of realistic and well-structured datasets of photos and videos. The diversity and richness of the datasets are crucial to benchmark the ML models and to evaluate their appropriateness to be applied in real-world digital forensics applications. An example is the development of third-party modules for the widely used Autopsy digital forensic application. This paper presents a dataset obtained by extracting a set of simple features from genuine and manipulated photos and videos, which are part of state-of-the-art existing datasets. The resulting dataset is balanced, and each entry comprises a label and a vector of numeric values corresponding to the features extracted through a Discrete Fourier Transform (DFT). The dataset is available in a GitHub repository, and the total amount of photos and video frames is 40,588 and 12,400, respectively. The dataset was validated and benchmarked with deep learning Convolutional Neural Networks (CNN) and Support Vector Machines (SVM) methods; however, a plethora of other existing ones can be applied. Generically, the results show a better F1-score for CNN when comparing with SVM, both for photos and videos processing. CNN achieved an F1-score of 0.9968 and 0.8415 for photos and videos, respectively. Regarding SVM, the results obtained with 5-fold cross-validation are 0.9953 and 0.7955, respectively, for photos and videos processing. A set of methods written in Python is available for the researchers, namely to preprocess and extract the features from the original photos and videos files and to build the training and testing sets. Additional methods are also available to convert the original PKL files into CSV and TXT, which gives more flexibility for the ML researchers to use the dataset on existing ML frameworks and tools.

show abstract

“…Recent deep-learning models have been developed to tackle the task of forgery detection [ 13 ]. These methods can be trained to detect specific falsification techniques such as splicing [ 14 , 15 ], copy-move [ 16 , 17 ] and inpainting [ 18 , 19 ], or to detect general attacks [ 20 , 21 , 22 ].…”

Section: Introductionmentioning

confidence: 99%

Forgery Detection in Digital Images by Multi-Scale Noise Estimation

et al. 2021

View full text Add to dashboard Cite

A complex processing chain is applied from the moment a raw image is acquired until the final image is obtained. This process transforms the originally Poisson-distributed noise into a complex noise model. Noise inconsistency analysis is a rich source for forgery detection, as forged regions have likely undergone a different processing pipeline or out-camera processing. We propose a multi-scale approach, which is shown to be suitable for analyzing the highly correlated noise present in JPEG-compressed images. We estimate a noise curve for each image block, in each color channel and at each scale. We then compare each noise curve to its corresponding noise curve obtained from the whole image by counting the percentage of bins of the local noise curve that are below the global one. This procedure yields crucial detection cues since many forgeries create a local noise deficit. Our method is shown to be competitive with the state of the art. It outperforms all other methods when evaluated using the MCC score, or on forged regions large enough and for colorization attacks, regardless of the evaluation metric.

show abstract

A Comprehensive Review of Deep-Learning-Based Methods for Image Forensics

Cited by 48 publications

References 148 publications

Exposing Manipulated Photos and Videos in Digital Forensics Analysis

Exposing Manipulated Photos and Videos in Digital Forensics Analysis

A Dataset of Photos and Videos for Digital Forensics Analysis Using Machine Learning Processing

Forgery Detection in Digital Images by Multi-Scale Noise Estimation

Contact Info

Product

Resources

About