Feature-Based Explanations Don't Help People Detect Misclassifications of Online Toxicity

Abstract

We present an experimental assessment of the impact of feature attribution-style explanations on human performance in predicting the consensus toxicity of social media posts with advice from an unreliable machine learning model. By doing so we add to a small but growing body of literature inspecting the utility of interpretable machine learning in terms of human outcomes. We also evaluate interpretable machine learning for the first time in the important domain of online toxicity, where fully-automated methods have faced criticism as being inadequate as a measure of toxic behavior.We find that, contrary to expectations, explanations have no significant impact on accuracy or agreement with model predictions, through they do change the distribution of subject error somewhat while reducing the cognitive burden of the task for subjects. Our results contribute to the recognition of an intriguing expectation gap in the field of interpretable machine learning between the general excitement the field has engendered and the ambiguous results of recent experimental work, including this study.

Evaluating the Utility of Conformal Prediction Sets for AI-Advised Image Labeling

“…Withholding predictions of an uncalibrated model may improve decision quality. Consistent with prior work on AI-advised decision-making (e.g., [7,10,21,38]), our results suggest that when a model is well-calibrated and more accurate than humans alone, users with access to its predictions can perform better than without the model, but not as well as the model alone for easier instances. When the model is poorly calibrated, the type of prediction display affects whether people can perform better by accessing the model predictions.…”

A Meta-Analysis of the Utility of Explainable Artificial Intelligence in Human-AI Decision-Making

“…In recent years, there has been a surge of research in human-AI decision-making, with a growing number of studies conducting behavioral experiments to gain a better understanding of how humans form decisions in the presence of AI (Alufaisan et al n.d.;Buçinca et al 2020;Carton et al 2020;Lai et al 2020;Lai and Tan 2019;Liu et al 2021;Zhang et al 2020). This research has focused on improving human-AI decisionmaking to optimize team performance (Buçinca et al 2020;Zhang et al 2020).…”

Human-AI Collaboration: The Effect of AI Delegation on Human Task Performance and Task Satisfaction

“…Thus, the AI model often provides the confidence level of the decision [51,75] or an additional explanation for its decision [1,42]. Several works have evaluated whether different types of explanations can support humans' understanding of the AI model so that they identify the right cases to rely on the recommendations [4,13,15,69]. Explanations can lead people to rely too much on the decision of the AI model, particularly when its suggestion is incorrect [6].…”