Segment and Complete: Defending Object Detectors against Adversarial Patch Attacks with Robust Patch Detection

Abstract

Object detection plays a key role in many security-critical systems. Adversarial patch attacks, which are easy to implement in the physical world, pose a serious threat to stateof-the-art object detectors. Developing reliable defenses for object detectors against patch attacks is critical but severely understudied. In this paper, we propose Segment and Complete defense (SAC), a general framework for defending object detectors against patch attacks through detecting and removing adversarial patches. We first train a patch segmenter that outputs patch masks that provide pixel-level localization of adversarial patches. We then propose a self adversarial training algorithm to robustify the patch segmenter. In addition, we design a robust shape completion algorithm, which is guaranteed to remove the entire patch from the images given the outputs of the patch segmenter are within a certain Hamming distance of the ground-truth patch masks. Our experiments on COCO and xView datasets demonstrate that SAC achieves superior robustness even under strong adaptive attacks with no performance drop on clean images, and generalizes well to unseen patch shapes, attack budgets, and unseen attack methods. Furthermore, we present the APRICOT-Mask dataset, which augments the APRICOT dataset with pixel-level annotations of adversarial patches. We show SAC can significantly reduce the targeted attack success rate of physical patch attacks.

Securing Tiny Transformer-Based Computer Vision Models: Evaluating Real-World Patch Attacks

“…Ji et al [32] adapt the YOLOv2 [24] network, introducing an additional patch class that the model can detect independently from the other objects in the image, thus effectively separating the valuable information from the malicious attack. In Segment and Complete [33], the authors introduce an additional stage in the object detection pipeline, firstly determining the adversarial patch location and masking it at the pixel level, followed by performing object detection on the so-cleaned image. Albeit empirically efficient, these methods are prone to be rendered ineffective by adaptive attackers, as shown by Chiang et al [34] in the context of image classification.…”