Quantifying Feature Contributions to Overall Disparity Using Information Theory

Abstract

When a machine-learning algorithm makes biased decisions, it can be helpful to understand the "sources" of disparity to explain why the bias exists. Towards this, we examine the problem of quantifying the contribution of each individual feature to the observed disparity. If we have access to the decision-making model, one potential approach (inspired from intervention-based approaches in explainability literature) is to vary each individual feature (while keeping the others fixed), and use the resulting change in disparity to quantify its contribution. However, we may not have access to the model or be able to test/audit its outputs for individually varying features. Furthermore, the decision may not always be a deterministic function of the input features (e.g., with human-in-the-loop). For these situations, we might need to explain contributions using purely distributional (i.e., observational) techniques, rather than interventional. We ask the question: what is the "potential" contribution of each individual feature to the observed disparity in the decisions when the exact decision-making mechanism is not accessible? We first provide canonical examples (thought experiments) that help illustrate the difference between distributional and interventional approaches to explaining contributions, and when either is better suited. When unable to intervene on the inputs, we quantify the "redundant" statistical dependency about the protected attribute that is present in both the final decision and an individual feature, by leveraging a body of work in information theory called Partial Information Decomposition. We also perform a simple case study to show how this technique could be applied to quantify contributions.

REFRESH: Responsible and Efficient Feature Reselection guided by SHAP values

“…However, this calculation will not be accurate and outcomes can significantly differ from true model outcomes i.e. when feature 𝑎 is not used to train the model [13,19,24,33]. In fact, (interventional) SHAP simulates the removal of features by marginalising over their marginal distributions and not by re-training a new model without such features [36].…”

A Review of Partial Information Decomposition in Algorithmic Fairness and Explainability

“…In Section 3, we first introduce the problem setup for quantifying non-exempt disparity as discussed in [8,29], and present several canonical examples and candidate measures, understanding their pros and cons, until we arrive at the proposed measure in [8] that satisfies the desirable properties. In Section 4, we review how PID can help in assessing the contributions of either features or data points with applications in feature selection (as discussed in [31]). Related works include [35][36][37][38].…”

“…Towards answering this question, ref. [31] proposes two measures for quantifying the contribution of each feature to the overall disparity. The first measure, which is referred to as interventional contribution, is defined as follows:…”

“…Instead, ref. [31] focuses on introducing information-theoretic measures to examine the problem from a distributional lens, and contrasting contribution and potential contribution, also touching upon the issue of substitute features. We include a summary in Table 2.…”

“…We include a summary in Table 2. We also refer to [31] for a more detailed discussion. Accounts for redundant features that can substitute an important feature when it is dropped.…”