SketchEdit: Mask-Free Local Image Manipulation with Partial Sketches

Zeng, Yu; Lin, Zhe; Patel, Vishal M.

doi:10.48550/arxiv.2111.15078

Cited by 1 publication

(3 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They tend to detect manipulations in all images, resulting in near 0.5 AUC and 0.0 F1 scores. [44,20,44,39,59,58,61] do not necessarily generate pixel-level masks during their editing process, existing fully-supervised methods cannot make use of these weakly-labeled data. We investigate the capacity of fully-supervised methods and our weakly-supervised method on two additional datasets [43,45], which contain manipulations that are different from the standard setting (i.e., copy-move, splicing, and inpainting).…”

Section: Comparison With the State-of-the-artmentioning

confidence: 99%

“…With the advent of increasingly powerful image editing techniques [39,24,26,56,36,20,59,58,61,20,44], image manipulation has never been so convenient, and can be easily accomplished using natural language [44,20,44] or sketch [39,59,58,61] by general users. Such advances allow malicious users to easily manipulate images, creating fake news, promoting blackmail, and generating Deep- (a) In-distribution (IND) manipulation detection result on the testing split of the CASIA dataset [10,11].…”

Section: Introductionmentioning

confidence: 99%

“…Although the pixel-mask can provide full supervision to help the model differentiate authentic and tampered image regions, the cost of such image annotation is non-trivial and limits the amount of the training images. On the other hand, emerging language-driven image editing/synthesis or sketch-based manipulation methods do not necessarily generate pixel-level masks during the editing process [39,59,58,61,44,20,44,36], but still have great potential to help train image manipulation detection if properly used.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Language-guided Human Motion Synthesis with Atomic Actions

Zhai,

Huang,

Luan

et al. 2023

Proceedings of the 31st ACM International Conference on Multimedia

View full text Add to dashboard Cite

As advanced image manipulation techniques emerge, detecting the manipulation becomes increasingly important. Despite the success of recent learning-based approaches for image manipulation detection, they typically require expensive pixel-level annotations to train, while exhibiting degraded performance when testing on images that are differently manipulated compared with training images. To address these limitations, we propose weakly-supervised image manipulation detection, such that only binary imagelevel labels (authentic or tampered with) are required for training purpose. Such a weakly-supervised setting can leverage more training images and has the potential to adapt quickly to new manipulation techniques. To improve the generalization ability, we propose weakly-supervised selfconsistency learning (WSCL) to leverage the weakly annotated images. Specifically, two consistency properties are learned: multi-source consistency (MSC) and inter-patch consistency (IPC). MSC exploits different content-agnostic information and enables cross-source learning via an online pseudo label generation and refinement process. IPC performs global pair-wise patch-patch relationship reasoning to discover a complete region of manipulation. Extensive experiments validate that our WSCL, even though is weakly supervised, exhibits competitive performance compared with fully-supervised counterpart under both in-distribution and out-of-distribution evaluations, as well as reasonable manipulation localization ability.

show abstract