Removing Spurious Features can Hurt Accuracy and Affect Groups Disproportionately

Abstract

The presence of spurious features interferes with the goal of obtaining robust models that perform well across many groups within the population. A natural remedy is to remove spurious features from the model. However, in this work we show that removal of spurious features can decrease accuracy due to the inductive biases of overparameterized models. We completely characterize how the removal of spurious features affects accuracy across different groups (more generally, test distributions) in noiseless overparameterized linear regression. In addition, we show that removal of spurious feature can decrease the accuracy even in balanced dataset-each target co-occurs equally with each spurious feature; and it can inadvertently make the model more susceptible to other spurious features. Finally, we show that robust self-training can remove spurious features without affecting the overall accuracy. Experiments on the Toxic-Comment-Detectoin and CelebA datasets show that our results hold in non-linear models.

Data and its (dis)contents: A survey of dataset development and use in machine learning research

“…In addition, while a variety of recent papers have proposed methods for removing spurious cues from training data or ''de-biasing'' models, recent work has shown that this can be damaging for model accuracy. 88 In contrast to a focus on statistical properties of datasets as a site for addressing and mitigating harms, Denton et al 89 propose a research agenda in the ''data genealogy'' paradigm that promotes critical assessment of the design choices with respect to the data sources, theoretical motivations, and methods used for constructing datasets. Prospective accounting for dataset contents using some of the methods discussed at the end of the previous section can offset the potential of post-hoc documentation debt that can be incurred otherwise.…”

Understanding and Improving Robustness of Vision Transformers through Patch-based Negative Augmentation

“…Recently, Vision Transformers (ViTs, Dosovitskiy et al (2021)) sparked great interest in the literature, as a radically new model architecture offering significant accuracy improvements and with hope of new robustness benefits. Over the past decade, there has been extensive work on understanding the robustness of convolution-based neural architectures, as the dominant design for visual tasks; researchers have explored adversarial robustness (Szegedy et al, 2013), domain generalization (Xiao et al, 2021;Khani & Liang, 2021), feature biases (Brendel & Bethge, 2019;Geirhos et al, 2018;Hermann et al, 2020). As a result, with the new promise of vision transformers, it is critical to understand their properties and in particular their robustness.…”

Data collection and quality challenges for deep learning

“…Fairness-oriented [133,80,59,79] Robustness-oriented [144,72,136] Equal Mergers [110,112,132,122] Table 1: Taxonomy of data collection and quality techniques for deep learning.…”

“…After training, the classifier's output becomes independent of the sensitive group. Fair training without spurious features [72] addresses the problem of preventing feature removal from being discriminating. A self-training technique is proposed to mitigate accuracy degradation and biased effects (Figure 32).…”