Solving Explainability Queries with Quantification: The Case of Feature Relevancy

Abstract

Trustable explanations of machine learning (ML) models are vital in high-risk uses of artificial intelligence (AI). Apart from the computation of trustable explanations, a number of explainability queries have been identified and studied in recent work. Some of these queries involve solving quantification problems, either in propositional or in more expressive logics. This paper investigates one of these quantification problems, namely the feature relevancy problem (FRP), i.e.\ to decide whether a (possibly sensitive) feature can occur in some explanation of a prediction. In contrast with earlier work, that studied FRP for specific classifiers, this paper proposes a novel algorithm for the \fprob quantification problem which is applicable to any ML classifier that meets minor requirements. Furthermore, the paper shows that the novel algorithm is efficient in practice. The experimental results, obtained using random forests (RFs) induced from well-known publicly available datasets, demonstrate that the proposed solution outperforms existing state-of-the-art solvers for Quantified Boolean Formulas (QBF) by orders of magnitude. Finally, the paper also identifies a novel family of formulas that are challenging for currently state-of-the-art QBF solvers.

Prediction Models for Glaucoma in a Multicenter Electronic Health Records Consortium: The Sight Outcomes Research Collaborative

“…Finally, although Shapley values are a convenient mathematical technique for understanding model behavior to some extent, it is crucial to acknowledge that there are limitations to this technique, including prior studies that have suggested that results can sometimes be inaccurate and misleading. 21 , 22 Therefore, it is still essential to consider explainability results in the context of prior similar studies and overall established medical knowledge.…”

Delivering Trustworthy AI through Formal XAI

“…Explanability queries -feature relevancy [32], [34], [35]. Let us consider a classifier M, with features F , domains D i , i ∈ F , classes K, a classification function κ : F → K, and a concrete instance (v, c), v ∈ F, c ∈ K.…”