Data-centric explainability and generating complex stories as explanations from machine learning models

Abstract

One of the major limitations of most trained machine learning (ML) models is the lack of explainability, which makes them incomprehensible to humans. Particularly in the healthcare field, individual nuances can have serious implications for diagnosis, particularly from the viewpoint of treatment [1]. Therefore, it is important to ensure that model predictions are accurate. Furthermore, most people working in the healthcare field do not possess computer knowledge. They are not able to read the model's conclusions or predictions directly. Consequently, it is also necessary to translate the prediction results into a readable narrative. Our research focuses on the healthcare domain and compares and contrasts two main methods in explainable AI: class-contrastive explanations and TREPAN (tree-based explanations). The goal of this study is to generate complex stories (diagnoses) from both single-patient and group-patient perspectives in order to make predictions more understandable for a wider audience of patients. Class-contrastive techniques are widely used in various fields, especially in the healthcare field. In this method, class-contrastive counterfactual reasoning is applied to a machine learning model on tabular data features, observing the changes in prediction by altering the inputs and estimating the effects of a change in variables on the results [1]. This approach displays prediction results as heatmaps and hierarchical clustering visualisations, which are easier for ordinary people to follow and provide clearer visual and textual explanations. Aside from that, TREPAN is also an approach for extracting the neural network, increasing the explainability. It is a method for extracting symbolic neural network representations from a decision tree, including the concept represented in the network [17]. Due to its broader applicability, this algorithm is suitable for networks with larger dimensions and high-dimensional inputs [17]. Most importantly, both methods provide a visual representation of the prediction model. We will extend the feature from tabular data to numeric data, using class-contrastive techniques to generate a decision tree, compare it with TREPAN's tree, and reconcile conclusions obtained from both. The method in this work extracts from machine learning algorithms a complex explanation that can be interpreted by humans.