Deep Convolutional Neural Network for Identifying Seam-Carving Forgery

Abstract

Seam carving is a representative content-aware image retargeting approach to adjust the size of an image while preserving its visually prominent content. To maintain visually important content, seam-carving algorithms first calculate the connected path of pixels, referred to as the seam, according to a defined cost function and then adjust the size of an image by removing and duplicating repeatedly calculated seams. Seam carving is actively exploited to overcome diversity in the resolution of images between applications and devices; hence, detecting the distortion caused by seam carving has become important in image forensics. In this paper, we propose a convolutional neural network (CNN)-based approach to classifying seamcarving-based image retargeting for reduction and expansion. To attain the ability to learn low-level features, we designed a CNN architecture comprising five types of network blocks specialized for capturing subtle signals. An ensemble module is further adopted to both enhance performance and comprehensively analyze the features in the local areas of the given image. To validate the effectiveness of our work, extensive experiments based on various CNN-based baselines were conducted. Compared to the baselines, our work exhibits state-of-the-art performance in terms of three-class classification (original, seam inserted, and seam removed). In addition, our model with the ensemble module is robust for various unseen cases. The experimental results also demonstrate that our method can be applied to localize both seam-removed and seam-inserted areas.

A comparison study to detect seam carving forgery in JPEG images with deep learning models

“…Optimization needs to be performed during detections [55][56][57][58] . Other seam-carving-based detection includes but not limited to [59][60][61][62] .…”

Data-driven personalisation of television content: a survey

“…cropping and warping [184], or seam carving and cropping [185]. It is worth noting that when applying warping or seam carving to the frames of a video, apart from undesirable artefacts introduced [186], the original video's semantic content might be distorted significantly [187]. In [188] it is argued that cropping methods are more suitable for video aspect ratio transformation when the minimization of semantic distortions is a prerequisite, as they select a region of interest in the video frames but do so without introducing any distortion to the visual content.…”

A Fast Smart-Cropping Method and Dataset for Video Retargeting

“…cropping and warping [9], or seam carving and cropping [10]). It is easily understood that when applying warping or seam carving to the frames of a video, apart from undesirable artifacts introduced [11], the original video content is distorted significantly [12]. For example, let us consider a video frame depicting two persons at the edges of the image with the content between them being a uniform background.…”