Image Splicing Localization using a Multi-task Fully Convolutional Network (MFCN)

Abstract

In this work, we propose a technique that utilizes a fully convolutional network (FCN) to localize image splicing attacks. We first evaluated a single-task FCN (SFCN) trained only on the surface label. Although the SFCN is shown to provide superior performance over existing methods, it still provides a coarse localization output in certain cases. Therefore, we propose the use of a multi-task FCN (MFCN) that utilizes two output branches for multi-task learning. One branch is used to learn the surface label, while the other branch is used to learn the edge or boundary of the spliced region. We trained the networks using the CASIA v2.0 dataset, and tested the trained models on the CASIA v1.0, Columbia Uncompressed, Carvalho, and the DARPA/NIST Nimble Challenge 2016 SCI datasets. Experiments show that the SFCN and MFCN outperform existing splicing localization algorithms, and that the MFCN can achieve finer localization than the SFCN.

Learning Rich Features for Image Manipulation Detection

“…Cozzolino et al [9] treat this problem as an anomaly detection task and use an autoencoder based on extracted features to distinguish those regions that are difficult to reconstruct as tampered regions. Salloum et al [29] use a Fully Convolutional Network (FCN) framework to directly pre-dict the tampering mask given an image. They also learn a boundary mask to guide the FCN to look at tampered edges, which assists them in achieving better performance in various image manipulation datasets.…”

“…To fine-tune our model on these datasets, we extract the bounding box from the ground-truth mask. We compare with other approaches on the same training and testing split protocol as [2] (for NIST16 and COVER) and [29] (for Columbia and CASIA). See Table 2.…”

“…We use pixel level F 1 score and Area Under the receiver operating characteristic Curve (AUC) as our evaluation metrics for performance comparison. F 1 score is a pixel level evaluation metric for image manipulation detection, as discussed in [33,29]. We vary different thresholds and use the highest F 1 score as the final score for each image, which follows the same protocol in [33,29].…”

“…F 1 score is a pixel level evaluation metric for image manipulation detection, as discussed in [33,29]. We vary different thresholds and use the highest F 1 score as the final score for each image, which follows the same protocol in [33,29]. We assign the confidence score to every pixel in the detected bounding boxes for pixel-level AUC evaluation.…”

“…• MFCN: A multi-task edge-enhanced FCN based network [29] jointly detecting tampered edges using edge binary masks and tampered regions using tampered region masks.…”