ERASER: A Benchmark to Evaluate Rationalized NLP Models

Abstract

State-of-the-art models in NLP are now predominantly based on deep neural networks that are opaque in terms of how they come to make predictions. This limitation has increased interest in designing more interpretable deep models for NLP that reveal the 'reasoning' behind model outputs. But work in this direction has been conducted on different datasets and tasks with correspondingly unique aims and metrics; this makes it difficult to track progress. We propose the Evaluating Rationales And Simple English Reasoning (ERASER ) benchmark to advance research on interpretable models in NLP. This benchmark comprises multiple datasets and tasks for which human annotations of "rationales" (supporting evidence) have been collected. We propose several metrics that aim to capture how well the rationales provided by models align with human rationales, and also how faithful these rationales are (i.e., the degree to which provided rationales influenced the corresponding predictions). Our hope is that releasing this benchmark facilitates progress on designing more interpretable NLP systems. The benchmark, code, and documentation are available at https://www.eraserbenchmark.com/ Commonsense Explanations (CoS-E)Where do you find the most amount of leafs? (a) Compost pile (b) Flowers (c) Forest (d) Field (e) Ground Movie ReviewsIn this movie, … Plots to take over the world. The acting is great! The soundtrack is run-of-the-mill, but the action more than makes up for it (a) Positive (b) Negative Evidence InferenceArticle Patients for this trial were recruited … Compared with 0.9% saline, 120 mg of inhaled nebulized furosemide had no effect on breathlessness during exercise. (a) Sig. decreased (b) No sig. difference (c) Sig. increased Prompt With respect to breathlessness, what is the reported difference between patients receiving placebo and those receiving furosemide? e-SNLI H A man in an orange vest leans over a pickup truck P A man is touching a truck (a) Entailment (b) Contradiction (c) Neutral

Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?

“…Relation to Automatic Tests. Prior works have proposed automatic metrics for feature importance estimates (Nguyen, 2018;Hooker et al, 2019;DeYoung et al, 2020). Typically these operate by checking that model behavior follows reasonable patterns on counterfactual inputs constructed using the explanation, e.g., by masking "important" features and checking that a class score drops.…”

Why do you think that? Exploring Faithful Sentence-Level Rationales Without Supervision

“…We aim at faithful explanations -the identification of the actual reason for the model's prediction, which is essential for accountability, fairness, and credibility (Chakraborty et al, 2017;Wu and Mooney, 2019) to evaluate whether a model's prediction is based on the correct evidence. The recently published ERASER benchmark (DeYoung et al, 2020) provides multiple datasets with annotated rationales, i.e., parts of the input document, which are essential for correct predictions of the target variable (Zaidan et al, 2007). By contrast to post-hoc techniques to identify relevant input parts such as LIME (Ribeiro et al, 2016) or input reduction (Feng et al, 2018), we focus on models that are faithful by design, in which the selected rationale matches the full underlying evidence used for the prediction.…”

“…Existing strategies mostly rely on REINFORCE (Williams, 1992) style learning (Lei et al, 2016; or on training two disjoint models (Lehman et al, 2019;DeYoung et al, 2020), in the latter case depending on rationale supervision. This poses critical limitations as rationale annotations are costly to obtain and, in many cases, not available.…”

“…Understanding the deep neural networks' decisions has gained increasing interest in the research community (DeYoung et al, 2020;Alishahi et al, 2019;Jacovi and Goldberg, 2020). Several works are concerned with post-hoc techniques to explain decisions of blackbox models (Ribeiro et al, 2016;Feng et al, 2018;Camburu et al, 2019).…”

The Explanation Game: Towards Prediction Explainability through Sparse Communication

“…How should these different approaches be compared? Several diagnostic tests have been proposed: Jain and Wallace (2019) assessed the explanatory power of attention weights by measuring their correlation with input gradients; Wiegreffe and Pinter (2019) and DeYoung et al (2020) developed more informative tests, including a combination of comprehensiveness and sufficiency metrics and the correlation with human rationales; Jacovi and Goldberg (2020) proposed a set of evaluation recommendations and a graded notion of faithfulness. Most proposed frameworks rely on correlations and counterfactual simulation, sidestepping the main practical goal of prediction explainability-the ability to communicate an explanation to a human user.…”