FIND: Human-in-the-Loop Debugging Deep Text Classifiers

Abstract

Since obtaining a perfect training dataset (i.e., a dataset which is considerably large, unbiased, and well-representative of unseen cases) is hardly possible, many real-world text classifiers are trained on the available, yet imperfect, datasets. These classifiers are thus likely to have undesirable properties. For instance, they may have biases against some sub-populations or may not work effectively in the wild due to overfitting. In this paper, we propose FINDa framework which enables humans to debug deep learning text classifiers by disabling irrelevant hidden features. Experiments show that by using FIND, humans can improve CNN text classifiers which were trained under different types of imperfect datasets (including datasets with biases and datasets with dissimilar traintest distributions).

Towards Benchmarking the Utility of Explanations for Model Debugging

“…Improvements can be applied by retraining and either removing input features (Ribeiro et al, 2016) or integrating explanation annotations into the objective function via explanation regularization (Ross et al, 2017;Liu and Avci, 2019;Rieger et al, 2020). Alternatively, features can also be disabled on the representation level (Lertvittayakumjorn et al, 2020).…”

Proceedings of the First Workshop on Trustworthy Natural Language Processing

“…Improvements can be applied by retraining and either removing input features or integrating explanation annotations into the objective function via explanation regularization (Ross et al, 2017;Liu and Avci, 2019;Rieger et al, 2020). Alternatively, features can also be disabled on the representation level (Lertvittayakumjorn et al, 2020).…”

HILDIF: Interactive Debugging of NLI Models Using Influence Functions

“…Further, example-based methods, such as influence functions (Koh and Liang, 2017), identify training data points which are the most important for particular predictions. Existing works have proposed ways to improve models by incorporating human feedback, in response to the explanations, by: adding model constraints by fixing certain parameters (Stumpf et al, 2009;Lertvittayakumjorn et al, 2020), adding training samples (Teso and Kersting, 2019), and adjusting models' weights directly (Kulesza et al, 2015).…”

“…Recently, explanatory debugging has been applied to more complex models using refined interpretability methods. In FIND (Lertvittayakumjorn et al, 2020), a masking matrix is added at the end of a CNN text classifier so as to disable particular CNN filters based on human feedback in response to LRP-based explanations (Arras et al, 2016). In CAIPI (Teso and Kersting, 2019), the user investigates and corrects a LIMEbased explanation (Ribeiro et al, 2016) for each prediction.…”