Defending Against Image Corruptions Through Adversarial Augmentations

Abstract

Modern neural networks excel at image classification, yet they remain vulnerable to common image corruptions such as blur, speckle noise or fog. Recent methods that focus on this problem, such as AugMix and DeepAugment, introduce defenses that operate in expectation over a distribution of image corruptions. In contrast, the literature on p -norm bounded perturbations focuses on defenses against worst-case corruptions. In this work, we reconcile both approaches by proposing AdversarialAugment, a technique which optimizes the parameters of image-to-image models to generate adversarially corrupted augmented images. We theoretically motivate our method and give sufficient conditions for the consistency of its idealized version as well as that of DeepAugment. Our classifiers improve upon the state-ofthe-art on common image corruption benchmarks conducted in expectation on CIFAR-10-C and improve worst-case performance against p -norm bounded perturbations on both CIFAR-10 and IMAGENET.

Benchmarking Robustness of 3D Point Cloud Recognition Against Common Corruptions

“…(Koh et al, 2021) presented WILDS, a curated benchmark of 10 datasets reflecting a diverse range of distribution shifts that naturally arise in real-world applications. ; Cubuk et al (2018); Calian et al (2021) proposed augmentation methods to improve the corruption robustness in 2D vision tasks. On the adversarial robustness benchmarking front, Carlini et al (2019) discussed the methodological foundations, reviewed commonly accepted best practices, and suggested new methods for evaluating defenses to adversarial examples.…”

PRIME: A few primitives can boost robustness to common corruptions

“…Besides, there are cases where offline augmentation is not feasible as it relies on pre-trained or generative models which are unavailable in certain scenarios, e.g. DeepAugment [20] or AdA [6] cannot be applied on C-100. On the other hand, off-line augmentation may be necessary to avoid the computational cost of generating augmentations during training.…”

“…Although AugMix attains significant gains on CIFAR-10-C, it does not perform well against sophisticated benchmarks like ImageNet-C. DeepAugment (DA) [20] addresses this issue and diversifies the space of augmentations by introducing distorted images computed by perturbing the weights of image-to-image networks. DA, combined with AugMix, achieves the current state-of-the-art on ImageNet-C. Other schemes include: (i) worst-case noise training [37], (ii) inducing shape bias through stylized images [17], (iii) adversarial counterparts of DeepAugment [6] and AugMix [43], (iv) pre-training and/or adversarial training [24,45], (v) constraining the total variation of convolutional layers [38] and (vi) learning the image information in the phase rather than amplitude [7]. Besides, Vision Transformers [15] have been shown to be more robust to common corruptions than standard CNNs [4,31].…”

“…All models are implemented in PyTorch [33] and are trained for 100 epochs using a cyclic learning rate schedule [40] with cosine annealing and a maximum learning rate of 0.2 unless stated otherwise. For IN, we fine-tune a regularly pretrained network 6 with a maximum learning rate of 0.01 following Hendrycks et al [20]. We use SGD optimizer with momentum factor 0.9 and Nesterov momentum [32].…”

“…Still, guaranteeing a good coverage over the whole space of such corruptions is hard. Hence, the current literature relies mostly on increasing the complexity of their training pipelines by combining independently developed data augmentation strategies, to hopefully increase diversity of augmentations and achieve state-of-the-art results on different common corruptions benchmarks [6,7,20,24,37,43]. This incremental research strategy, though, has come at a cost.…”