Impact of Data Preparation and CNN’s First Layer on Performance of Image Forensics: A Case Study of Detecting Colorized Images

Abstract

In the field of image forensics, many convolutional neural network (CNN)-based forensic methods have been proposed and generally achieved the state-of-the-art performance. However, some questions are worth studying and answering regarding the trustworthiness of such methods, including for example the appropriateness of the discriminative information automatically extracted by CNN and the generalization performance on "unseen" data during the testing phase. In this paper, we study these questions in the case of a specific forensic problem of distinguishing between natural images (NIs) and colorized images (CIs). Through a series of experiments, we analyze the impact of data preparation and setting of the first layer of a recent state-of-the-art CNN-based method on the detector's forensic performance, in particular the generalization capability. We obtain some interesting observations which can serve as useful hints for carrying out image forensics experiments. Moreover, we propose a very simple method to improve the generalization performance of colorized image detection by combining decision results from CNN models with different settings at the network's first layer.

Learn with diversity and from harder samples: Improving the generalization of CNN-Based detection of computer-generated images

“…To solve this generalization problem, in this work, we consider two aspects of CNN: network architecture and network training. Our network design is inspired by Quan et al' work [32] about the impact of CNN's first layer on forensic performance, where they proposed a simple criterion to combine the predictions of two independently trained networks for obtaining the final result. In our work, we design and implement a novel two-branch CNN model and apply different initialization strategy to the first layer of these two branches.…”

“…The standing point of our network design is to enrich the diversity of feature learning. Inspired by the observation in [32] and ensemble learning, we design a novel two-branch network to automatically and efficiently combine the kernels initialized with SRM (Spatial Rich Model) filters [33] and Gaussian random distribution in the beginning of network, and it can be trained in the standard In our network, from L2 to L4, we do not use any pooling operation so as to retain useful discriminative information, which also helps for improving the diversity between the two branches. In the second branch (B2 in Fig.…”

“…Following [14], we adopt the standard 10-crop testing [34]: given a testing sample, the network extracts five patches of 233 × 233 pixels (the center and four corner patches), flips these five patches Table 1: The classification performance (HTER, in %, lower is better) of different network architectures. "NcgNet Comb" is the combination result of predictions of "NcgNet" and "NcgNet SRM" according to the combination criterion used by [32]. For the sake of clarity, the results of generalization performance on "unknown" rendering engines are presented in italics.…”