Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective

Abstract

Deep neural networks (DNNs) often rely on easy-to-learn discriminatory features, or cues, that are not necessarily essential to the problem at hand. For example, ducks in an image may be recognized based on their typical background scenery, such as lakes or streams. This phenomenon, also known as shortcut learning, is emerging as a key limitation of the current generation of machine learning models. In this work, we introduce a set of experiments to deepen our understanding of shortcut learning and its implications. We design a training setup with several shortcut cues, named WCST-ML, where each cue is equally conducive to the visual recognition problem at hand. Even under equal opportunities, we observe that (1) certain cues are preferred to others, (2) solutions biased to the easy-to-learn cues tend to converge to relatively flat minima on the loss surface, and (3) the solutions focusing on those preferred cues are far more abundant in the parameter space. We explain the abundance of certain cues via their Kolmogorov (descriptional) complexity: solutions corresponding to Kolmogorov-simple cues are abundant in the parameter space and are thus preferred by DNNs. Our studies are based on the synthetic dataset DSprites and the face dataset UTKFace. In our WCST-ML, we observe that the inborn bias of models leans toward simple cues, such as color and ethnicity. Our findings emphasize the importance of active human intervention to remove the inborn model biases that may cause negative societal impacts.

Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations

“…Shah et al [87] show empirically that in certain scenarios neural networks can suffer from extreme simplicity bias and rely on simple spurious features, while ignoring the core features; in Section 4.2 we revisit these problems and provide further discussion. Hermann and Lampinen [38] and Jacobsen et al [45] also show synthetic and natural examples, where neural networks ignore relevant features, and Scimeca et al [86] explore which types of shortcuts are more likely to be learned. Kolesnikov and Lampert [50] on the other hand show that on realistic datasets core and spurious features can often be distinguished from the latent representations learned by a neural network in the context of object localization.…”

Shortcut Learning Through the Lens of Early Training Dynamics

“…While the distribution shift explains part of the story, we emphasize that what is equally important for shortcut learning is the difficulty of the spurious features themselves (see Fig- 1). Previous works like Shah et al (2020); Scimeca et al (2021) hint at this. But we take this line of thought further by viewing shortcut learning as a phenomenon that impacts the dataset difficulty, which can be captured by monitoring early training dynamics.…”

“…Rather only those spurious features that are easier than the core features are potential shortcuts (see Fig- 1). Previous works like Shah et al (2020); Scimeca et al (2021) hint at this by saying that DNNs are biased towards simple solutions, and Dagaev et al (2021) use the "too-good-to-be-true" prior to emphasize that simple solutions are unlikely to be valid across contexts. Veitch et al (2021) distinguish various model features using tools from causality and stress test the models for counterfactual invariance.…”

Learning Fair Classifiers with Partially Annotated Group Labels

“…Biases in machine learning. Emerging studies on DNNs have revealed that DNNs rely on shortcut biases [4,10,21,22,44]. The existing de-biasing methods let a model less attend on the dataset biases in an implicit way by using extra biased networks [4,10] or data augmentations [22] without using bias labels.…”