Topics to Avoid: Demoting Latent Confounds in Text Classification

Abstract

Despite impressive performance on many text classification tasks, deep neural networks tend to learn frequent superficial patterns that are specific to the training data and do not always generalize well. In this work, we observe this limitation with respect to the task of native language identification. We find that standard text classifiers which perform well on the test set end up learning topical features which are confounds of the prediction task (e.g., if the input text mentions Sweden, the classifier predicts that the author's native language is Swedish). We propose a method that represents the latent topical confounds and a model which "unlearns" confounding features by predicting both the label of the input text and the confound; but we train the two predictors adversarially in an alternating fashion to learn a text representation that predicts the correct label but is less prone to using information about the confound. We show that this model generalizes better and learns features that are indicative of the writing style rather than the content. 1

Unsupervised Discovery of Implicit Gender Bias

“…For the primary prediction models, we use the same architectures as Kumar et al (2019), including training multiple (2) adversaries. We perform minimal hyper-parameter tuning, primarily using the same parameters as Kumar et al (2019), with the exception of the learning rate, which we changed slightly to decrease fluctuations in validation accuracy, and the number of training epochs for each phase of the model, which we increased or decreased as needed based on how long the validation accuracy improved for. These changes were determined by manual tuning over < 10 trials.…”

“…In our data, while O TXT is observed, W TRAITS is not possible to match on (further discussion in §4). Instead, we use an adversarial objective drawn from Kumar et al (2019) to encourage the model to ignore W TRAITS.…”

“…In Figure 1, lo(Canada, Person 1) would be high, as COM TXT values addressed to Person 1 often contain the word Canada. Then, following Kumar et al (2019), we define a distribution: for all k ∈ OW and an input COM TXT i , = w 1 , . .…”

“…We refer to the confound vector as t i . Justification for the log-odds representation as opposed to alternatives is presented in Kumar et al (2019).…”

“…where U represents a uniform distribution, CE represents cross-entropy loss, and KL represents KLdivergence. We refer to Kumar et al (2019) for the training procedure that alternates minimizing this objective and training the adversary.…”