Towards a More Reliable Interpretation of Machine Learning Outputs for Safety-Critical Systems using Feature Importance Fusion

Abstract

When machine learning supports decision-making in safety-critical systems, it is important to verify and understand the reasons why a particular output is produced. Although feature importance calculation approaches assist in interpretation, there is a lack of consensus regarding how features' importance is quantified, which makes the explanations offered for the outcomes mostly unreliable. A possible solution to address the lack of agreement is to combine the results from multiple feature importance quantifiers to reduce the variance of estimates. Our hypothesis is that this will lead to more robust and trustworthy interpretations of the contribution of each feature to machine learning predictions. To assist test this hypothesis, we propose an extensible Framework divided in four main parts: (i) traditional data pre-processing and preparation for predictive machine learning models; (ii) predictive machine learning; (iii) feature importance quantification and (iv) feature importance decision fusion using an ensemble strategy. We also introduce a novel fusion metric and compare it to the state-of-the-art. Our approach is tested on synthetic data, where the ground truth is known. We compare different fusion approaches and their results for both training and test sets. We also investigate how different characteristics within the datasets affect the feature importance ensembles studied.

Mechanistic Interpretation of Machine Learning Inference: A Fuzzy Feature Importance Fusion Approach

“…Estimating the importance of features in ML predictive analytics is currently very unreliable. Different ML models, FI techniques, and subsets of data generate different importance coefficients, often with diverse magnitudes, for the same features [4]. These uncertainties in identifying the contribution of features to ML outputs are due to:…”

“…Zhai and Chen [24] improved ensemble FI by using multiple ML models and gains in gini importance defining the final FI. Rengasamy et al [4], proposed a model agnostic ensemble FI framework to improve FI quantification using multiple models and multiple FI calculation methods. They studied several crisp fusion metrics such as mean, median, majority vote, rank correlation, combination with majority vote, and modified Thompson tau test.…”

“…We validate our approach using synthetic data described by Rengasamy et al [4], as it allows us to verify the efficacy of feature importance produced by FEFI against the ground truth. Nine datasets are used for regression, with 2000 instances and different number of features, feature informative levels, feature noise levels, or feature interaction levels.…”

“…One way to explain ML models outputs is by post-training analysis, such as feature importance (FI) analysis. FI estimates the contribution of each feature (independent parameter) to the model output [4], [8]. Using FI to understand how decisions are made aids the establishment of true causality between important data attributes and outcomes in model inference [6], [9].…”

“…To address these uncertainties in FI, Rengasamy et al [4] proposed an ensemble feature importance (EFI) method. Here, multiple FI approaches are applied to a set of ML models and their crisp importance values are combined to produce a final importance for each feature.…”