We introduce a variable importance measure to quantify the impact of individual variables to a decision made by a black box function. Our measure is based on the Shapley value from cooperative game theory. Many measures of variable importance operate by changing some predictor values with others held fixed, and that usually creates unlikely or even impossible combinations. Our cohort refinement Shapley approach measures variable importance only using observed data points. Instead of changing the value of a predictor we include or exclude subjects similar to the target subject on that predictor to form a similarity cohort. Then we apply Shapley value to the cohort averages. We also introduce a game theoretic way to aggregate multiple explanations and we illustrate the method on real data sets (titanic and Boston housing).
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.