Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU models

Abstract

Recent studies indicate that NLU models are prone to rely on shortcut features for prediction, without achieving true language understanding. As a result, these models fail to generalize to real-world out-of-distribution data. In this work, we show that the words in the NLU training set can be modeled as a longtailed distribution. There are two findings: 1) NLU models have strong preference for features located at the head of the long-tailed distribution, and 2) Shortcut features are picked up during very early few iterations of the model training. These two observations are further employed to formulate a measurement which can quantify the shortcut degree of each training sample. Based on this shortcut measurement, we propose a shortcut mitigation framework LTGR, to suppress the model from making overconfident predictions for samples with large shortcut degree. Experimental results on three NLU benchmarks demonstrate that our long-tailed distribution explanation accurately reflects the shortcut learning behavior of NLU models. Experimental analysis further indicates that LTGR can improve the generalization accuracy on OOD data, while preserving the accuracy on in-distribution data. Input x Teacher model Softmax Softmax Ground truth y Smoothed Softmax Distill loss Student loss Overconfident prediction Long-tailed distribution Example of model paying high attention to features on the head (a) long-tailed observation (b) Mitigation framework Shortcut degree Student model Head Long tail Shortcut degree Data statistics Model behavior

Measuring and Improving Model-Moderator Collaboration using Uncertainty Estimation

“…This suggests that naively applying debiasing techniques may incur unexpected negative impacts on other aspects of the moderation system. Further research is needed into modeling approaches that can achieve robust performance both in prediction and in uncertainty calibration under data bias and distributional shift (Nam et al, 2020;Utama et al, 2020;Du et al, 2021;Yaghoobzadeh et al, 2021;Bao et al, 2021;Karimi Mahabadi et al, 2020).…”

ClidSum: A Benchmark Dataset for Cross-Lingual Dialogue Summarization

“…UP only requires the pre-trained model to restore the order of ut- terances in noisy dialogues rather than predicting new words. Such NLU-style task would make it easier for models to learn shortcuts, i.e., models tend to exploit dataset biases as shortcuts to make predictions, rather than learn the semantic understanding and reasoning (Geirhos et al, 2020;Du et al, 2021). For ablation studies of combining two or three tasks, please refer to Appendix C.…”

"Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classification

“…Alternatively, one can make strong assumptions about what a ground truth should be like and compare salience rankings with what is expected to be the ground truth. In this vein human reasoning (Poerner et al, 2018;, gradient information (Du et al, 2021), aggregated model internal representations (Atanasova et al, 2020), changes in predicted probabilities (DeYoung et al, 2020; or surrogate models (Ding and Koehn, 2021) all have been taken as a proxy for the ground truth when evaluating salience methods. Unfortunately, they also resulted in divergent recommendations so the question of what the ground truth is and which method to use remains open.…”