Scale-Adaptive Adversarial Patch Attack for Remote Sensing Image Aircraft Detection

Abstract

With the adversarial attack of convolutional neural networks (CNNs), we are able to generate adversarial patches to make an aircraft undetectable by object detectors instead of covering the aircraft with large camouflage nets. However, aircraft in remote sensing images (RSIs) have the problem of large variations in scale, which can easily cause size mismatches between an adversarial patch and an aircraft. A small adversarial patch has no attack effect on large aircraft, and a large adversarial patch will completely cover small aircraft so that it is impossible to judge whether the adversarial patch has an attack effect. Therefore, we propose the adversarial attack method Patch-Noobj for the problem of large-scale variation in aircraft in RSIs. Patch-Noobj adaptively scales the width and height of the adversarial patch according to the size of the attacked aircraft and generates a universal adversarial patch that can attack aircraft of different sizes. In the experiment, we use the YOLOv3 detector to verify the effectiveness of Patch-Noobj on multiple datasets. The experimental results demonstrate that our universal adversarial patches are well adapted to aircraft of different sizes on multiple datasets and effectively reduce the Average Precision (AP) of the YOLOv3 detector on the DOTA, NWPU VHR-10, and RSOD datasets by 48.2%, 23.9%, and 20.2%, respectively. Moreover, the universal adversarial patch generated on one dataset is also effective in attacking aircraft on the remaining two datasets, while the adversarial patch generated on YOLOv3 is also effective in attacking YOLOv5 and Faster R-CNN, which demonstrates the attack transferability of the adversarial patch.

Adversarial Patch Attacks and Defences in Vision-Based Tasks: A Survey

“…g Lu et al [61] empirically demonstrated Patch-Noobj's transferability in three scenarios: dataset-to-dataset, model-to-model, and joint both. Wang et al [76] also demonstrated the transferability of the invisibility patch across models and datasets.…”

Adversarial Patch Attacks and Defences in Vision-Based Tasks: A Survey

“…Karmon et al [30] conducted patch perturbations in both images and networks for which the latter was more effective though not physically attainable. While it was more effective, LaVAN lacked robustness across transformations First for classification tasks [37] and object detection [53] First GAN-based [54], physical patch [55], and to fool cameras [56], [57] First for attacking depth estimation [58] and image semantics [59] Data independent adversarial patch [60], and adaptive adversarial patch [61] DiAP [60], Patch-Noobj [61], RAP (IPatch) [59] in 2021 [37] in 2017…”

“…On the other hand, it helped eradicate the requirement of manual camouflage nets to cover assets, because bigger assets or larger numbers of them often made it infeasible. Moreover, Lu et al [61] proposed Patch-Noobj to adaptively scale the patch size based on the size of attacked aircraft, showing attack transferability across both models and datasets.…”

“…Adversarial patch [37] VGG16, VGG19, ResNet50, InceptionV3, Xception ImageNet Yes LaVAN [30] InceptionV3 [78] ImageNet No QP patch [62], [63] InceptionV3 [78] ImageNet [79] No PS-GAN a [54] VGG16, ResNet34, VY [80], and their variants ImageNet [81], GTSRB [82] Yes Local patch via DIP b [64] VGG19, InceptionV3 ImageNet [81] No DiAP [60] VGG16, VGG19 [83], ResNet50 [84], InceptionV3 [78], Xception [ YOLOv2 [87], Faster R-CNN [69] Seld-made data (indoors and outdoors) Yes Physical patch +PGD e [70] YOLOv3 [89] MS COCO 2014 [90] Yes Patch against person detectors [56] YOLOv2 [87] INRIA [91] e Yes DPAttack [71] YOLOv4 [92], Faster R-CNN [69] Alibaba f No RPAttack [72] YOLOv4 [92], Faster R-CNN [93] Alibaba f , Pascal VOC 2007 [94] No Patch against aerial detection [57] YOLOv2 DOTA [95] No Patch-Noobj [61] YOLOv3 [89], YOLOv5 [96], Faster R-CNN [69] DOTA [95], NWPU VHR-10 [97], RSOD [98] No g…”

Adversarial Attacks against a Satellite-borne Multispectral Cloud Detector

“…1, need only access to the environment imaged by the victim, whereas digital attacks need access to the victim's test images (e.g., in a memory buffer); in this sense, physical attacks have weaker operational requirements and the associated impact is more concerning. For aerial/satellite RGB imagery, physical attacks on a classifier [11], aircraft detectors [14,40] and a car detector [16] have been investigated but only [16] provided real-world physical test results. For aerial/satellite multi/hyperspectral imagery, our work is arguably the first to consider physical adversarial attacks.…”