THAT: Two Head Adversarial Training for Improving Robustness at Scale

Abstract

Many variants of adversarial training have been proposed, with most research focusing on problems with relatively few classes. In this paper, we propose Two Head Adversarial Training (THAT), a two-stream adversarial learning network that is designed to handle the large-scale manyclass ImageNet dataset. The proposed method trains a network with two heads and two loss functions; one to minimize feature-space domain shift between natural and adversarial images, and one to promote high classification accuracy. This combination delivers a hardened network that achieves state of the art robust accuracy while maintaining high natural accuracy on ImageNet. Through extensive experiments, we demonstrate that the proposed framework outperforms alternative methods under both standard and "free" adversarial training settings.

Robustness through Cognitive Dissociation Mitigation in Contrastive Adversarial Training

“…Both methods proposed a new way to generate adversarial examples based on the CL loss instead of regular label-based losses(e.g., cross-entropy). [28] proposed to train two encoders: clean trained and adversarially trained, and use two loss functions: contrastive loss to minimize feature inconsistency between natural and adversarial samples, and CE loss to promote high classification accuracy. The adversarial samples are used in the self-supervised feature representation training to achieve feature robustness.…”