A transformer–CNN for deep image inpainting forensics

Zhu, Xiaoyan; Lu, Junyan; Ren, Honghao; Wang, Hongquan; Sun, Biao

doi:10.1007/s00371-022-02620-0

Cited by 10 publications

(7 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the last years, the focus for detecting inpainting images is to apply higher and more complex machine learning models, with some strong feature extract mechanisms. For example in [82] [83] or [84], the authors suggested that the noise variance is disturbing the inpainted area, thus by applying three Resnet feature blocks in a multi-scale network, they obtained very good results. They use for testing the latest state-of-the-art deep inpainting methods, but they random masks for applying the inpainting methods, thus not a very realistic approach.…”

Section: Machine Learning-based Methodsmentioning

confidence: 99%

Image Inpainting Forgery Detection: A Review

Barglazan,

Brad,

Constantinescu

2023

Preprint

View full text Add to dashboard Cite

In recent years, significant advancements in the field of machine learning have influenced the domain of image restoration. While these technological advancements present prospects for improving the quality of images, they also present difficulties, particularly the proliferation of manipulated or counterfeit multimedia information on the internet. The objective of this paper is to provide a comprehensive review of existing inpainting algorithms and forgery detections, with a specific emphasis on techniques that are designed for the purpose of removing objects from digital images. In this study, we will examine various techniques encompassing conventional texture synthesis methods, as well as those based on neural networks. Furthermore, we will explore the artifacts associated with the identification of modified photos and present the artifacts frequently introduced by the inpainting procedure and assess the state-of-the-art technology for detecting such modifications. Lastly, we shall look at the available datasets and how the methods compare with each other. Having covered all of the above, the final outcome of this study is to provide a comprehensive perspective on the abilities and constraints to detect images for which an inpainting object removal method was applied.

show abstract

Section: Machine Learning-based Methodsmentioning

confidence: 99%

Image Inpainting Forgery Detection: A Review

Barglazan,

Brad,

Constantinescu

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…[76] 2021 A tweaked version of a VGG model architecture [77] 2022 A CNN model with a focus on detecting noise inconsistencies [78] 2022 A U-NET VGG model that adds an enhancement block of five filters (four SRM + Laplacian) to be able to better detect inpainted areas.…”

Section: Referencementioning

confidence: 99%

“…An interesting thing was that they did not compare their work with the other that claimed to be SOTA [79]. Also, the work based on noise inconsistencies [77] appears to surpass these approaches in terms of results but not in terms of speed and model complexity (the authors in [83] claimed to use a small network model with approximately 3 M parameters only).…”

mentioning

confidence: 98%

“…Some authors even went in the direction that suggests that based on noise patterns, they will be able to uniquely identify camera models. To name a few, some of the most cited works are [70][71][72][73][74][75][76][77]. For e.g., in [70], the authors suggested computing noise for non-overlapping blocks and then unifying regions that have similar noise-thus partitioning the image into areas with the same noise intensities.…”

mentioning

confidence: 99%

“…In the last years, the focus regarding detecting inpainting images is on applying higher and more complex machine learning models with some strong feature extraction mechanisms. For example, in [77,78] the authors suggested that the noise variance disturbs the inpainted area, and thus by applying three Resnet feature blocks in a multi-scale network, they obtained very good results. They are used for assessing the latest stateof-the-art deep inpainting methods, but they use random masks for applying inpainting methods and are thus not very realistic approaches.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Image Inpainting Forgery Detection: A Review

Barglazan,

Brad,

Constantinescu

2024

J. Imaging

View full text Add to dashboard Cite

In recent years, significant advancements in the field of machine learning have influenced the domain of image restoration. While these technological advancements present prospects for improving the quality of images, they also present difficulties, particularly the proliferation of manipulated or counterfeit multimedia information on the internet. The objective of this paper is to provide a comprehensive review of existing inpainting algorithms and forgery detections, with a specific emphasis on techniques that are designed for the purpose of removing objects from digital images. In this study, we will examine various techniques encompassing conventional texture synthesis methods as well as those based on neural networks. Furthermore, we will present the artifacts frequently introduced by the inpainting procedure and assess the state-of-the-art technology for detecting such modifications. Lastly, we shall look at the available datasets and how the methods compare with each other. Having covered all the above, the outcome of this study is to provide a comprehensive perspective on the abilities and constraints of detecting object removal via the inpainting procedure in images.

show abstract