On cognitive preferences and the plausibility of rule-based models

Abstract

It is conventional wisdom in machine learning and data mining that logical models such as rule sets are more interpretable than other models, and that among such rule-based models, simpler models are more interpretable than more complex ones. In this position paper, we question this latter assumption by focusing on one particular aspect of interpretability, namely the plausibility of models. Roughly speaking, we equate the plausibility of a model with the likeliness that a user accepts it as an explanation for a prediction. In particular, we argue that-all other things being equal-longer explanations may be more convincing than shorter ones, and that the predominant bias for shorter models, which is typically necessary for learning powerful discriminative models, may not be suitable when it comes to user acceptance of the learned models. To that end, we first recapitulate evidence for and against this postulate, and then report the results of an evaluation in a crowdsourcing study based on about 3,000 judgments. The results do not reveal a strong preference for simple rules, whereas we can observe a weak preference for longer rules in some domains. We then relate these results to well-known cognitive biases such as the conjunction fallacy, the representative heuristic, or the recogition heuristic, and investigate their relation to rule length and plausibility.

The Pragmatic Turn in Explainable Artificial Intelligence (XAI)

“…Freitas (2014) examines the pros and cons of decision trees, classification rules, decision tables, nearest neighbors, and Bayesian network classifiers with respect to their interpretability, and discusses how to improve the comprehensibility of classification models in general. More recently, Fürnkranz et al (2018) performed an experiment with 390 participants to question the idea that the likeliness that a user will accept a logical model such as rule sets as an explanation for a decision is determined by the simplicity of the model. Lage et al (2019) also explore the complexities of rule sets to find features that make them more interpretable, while Piltaver et al (2016) undertake a similar analysis in the case of classification trees.…”

What can the conjunction fallacy tell us about human reasoning?

“…Therefore, the common assumption in the data mining and machine learning literature that users always find simpler models easier to understand and more convincing to believe is not necessarily well-founded. For a similar conclusion that questions the predominant "simplicity bias" when the plausibility of rule-based models is at issue (intended as the likeliness that a user accept the model as an explanation for a prediction), see also Fürnkranz et al (2019). Second, the results of the Anemia scenario suggest that the perceived plausibility of a list of causal explanations does not depend on their probability given the available evidence as much as on their being supported by the available evidence.…”

What Can the Conjunction Fallacy Tell Us about Human Reasoning?

“…Therefore, the common assumption in the data mining and machine learning literature that users always find simpler models easier to understand and more convincing to believe is not necessarily well-founded. For a similar conclusion that questions the predominant "simplicity bias" when the plausibility of rule-based models is at issue (intended as the likeliness that a user accept the model as an explanation for a prediction), see also Fürnkranz et al (2019).…”