Evaluating Local Interpretable Model-Agnostic Explanations on Clinical Machine Learning Classification Models

Kumarakulasinghe, Nesaretnam Barr; Blomberg, Tobias; Liu, Jintai; Leao, Alexandra Saraiva; Papapetrou, Panagiotis

doi:10.1109/cbms49503.2020.00009

Cited by 42 publications

(32 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To the best of our knowledge, our study is the first to compare the performances of expert and lay people when evaluating explanation methods. Previously, publications either focused on only expert groups (Hase & Bansal, 2020;Kumarakulasinghe et al, 2020) or only laypeople (Schmidt & Biessmann, 2019;Alufaisan et al, 2020). Our experiment shows no significant difference between expert and lay participants in our task-both perform similarly well, and even better on natural images: a replication of our main finding.…”

Section: Discussionmentioning

confidence: 99%

Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualization

Borowski¹,

Zimmermann²,

Schepers³

et al. 2020

Preprint

View full text Add to dashboard Cite

Feature visualizations such as synthetic maximally activating images are a widely used explanation method to better understand the information processing of convolutional neural networks (CNNs). At the same time, there are concerns that these visualizations might not accurately represent CNNs' inner workings. Here, we measure how much extremely activating images help humans to predict CNN activations. Using a well-controlled psychophysical paradigm, we compare the informativeness of synthetic images (Olah et al., 2017) with a simple baseline visualization, namely exemplary natural images that also strongly activate a specific feature map. Given either synthetic or natural reference images, human participants choose which of two query images leads to strong positive activation. The experiment is designed to maximize participants' performance, and is the first to probe intermediate instead of final layer representations. We find that synthetic images indeed provide helpful information about feature map activations (82±4% accuracy; chance would be 50%). However, natural images-originally intended to be a baseline-outperform synthetic images by a wide margin (92%±2% accuracy). Additionally, participants are faster and more confident for natural images, whereas subjective impressions about the interpretability of feature visualization are mixed. The higher informativeness of natural images holds across most layers, for both expert and lay participants as well as for hand-and randomly-picked feature visualizations. Even if only a single reference image is given, synthetic images provide less information than natural images (65 ± 5% vs. 73 ± 4%). In summary, popular synthetic images from feature visualizations are significantly less informative for assessing CNN activations than natural images. We argue that future visualization methods should improve over this simple baseline.

show abstract

Section: Discussionmentioning

confidence: 99%

Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualization

Borowski¹,

Zimmermann²,

Schepers³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Individual explanations for model behavior were provided for transparency into the output using local-interpretable model-agnostic explanations (LIME). 18 Decision curve analysis was used to determine the benefit of implementing the predictive algorithm in practice. The curve plots net benefit against the predicted probabilities of the outcome of interest, in this case, overnight admission, and provides the cost-benefit ratio for every value of the predicted probability.…”

Section: Methodsmentioning

confidence: 99%

Machine Learning Model Identifies Increased Operative Time and Greater BMI as Predictors for Overnight Admission After Outpatient Hip Arthroscopy

Song

Wilbur

et al. 2021

Arthroscopy, Sports Medicine, and Rehabilitation

View full text Add to dashboard Cite

Purpose The purposes of this study were to identify patient characteristics and risk factors for overnight admission following outpatient hip arthroscopy and to develop a machine learning algorithm that can effectively identify patients requiring admission following elective hip arthroscopy. Methods A retrospective review of a prospectively collected national surgical outcomes database was performed to identify patients who underwent elective outpatient hip arthroscopy from 2006 to 2018. Patients admitted overnight postoperatively were identified as those with length of stay of 1 or more days. Models were generated using random forest (RF), extreme gradient boosting (XGBoost), adaptive boosting (AdaBoost), elastic net penalized logistic regression, and an additional model was produced as a weighted ensemble of the four final algorithms. Results Overall, 1,276 patients were included. The median age was 43 years, and 64.2% (819) were female. Of the included patients, 109 (8.5%) required an overnight stay following elective outpatient hip arthroscopy. The most important factors for inpatient admission were increasing operative time, general anesthesia, age extremes, male gender, greater body mass index (BMI), American Society of Anesthesiologists classification >1, and the following preoperative lab values outside of normal ranges: sodium, platelet count, hematocrit, and leukocyte count. The ensemble model achieved the best performance based on discrimination assessed via internal validation (area under the curve = .71), calibration, and decision curve analysis. The model was integrated into a Web-based open-access application able to provide both personalized predictions and explanations. Conclusion A machine learning algorithm developed based on preoperative features identified increasing operative time, age extremes, greater BMI, sodium, hematocrit, platelets, and leukocyte count as the most important variables associated with inpatient admission with fair validity.

show abstract

“…Thus, it allows any complex classifier to be explained more simply in the clinical setting. LIME, can determine how much each variable in the data contributes to each estimate (patient specific) in the model [27]. Using the LIME method, it can be determined which variables affect each estimation in the model to what degree and in which direction, or which variable has more influence on the results of each estimation in the model compared to other variables.…”

Section: 3localmentioning

confidence: 99%

Detection of risk factors of PCOS patients with Local Interpretable Model-agnostic Explanations (LIME) Method that an explainable artificial intelligence model

ÇİÇEK

Küçükakçali

Yağın

2021

The Journal of Cognitive Systems

View full text Add to dashboard Cite

Aim: In this study, it is aimed to extract patient-based explanations of the contribution of important features in the decision-making process (estimation) of the Random forest (RF) model, which is difficult to interpret for PCOS disease risk, with Local Interpretable Model-Agnostic Explanations (LIME). Materials and Methods: In this study, the Local Interpretable Model-Agnostic Annotations (LIME) method was applied to the “Polycystic ovary syndrome” dataset to explain the Random Forest (RF) model, which is difficult to interpret for PCOS risk factors estimation. This dataset is available at https://www.kaggle.com/prasoonkottarathil/polycystic-ovary-syndrome-pcos. Results: Accuracy, sensitivity, specificity, positive predictive value, negative predictive value and balanced accuracy obtained from the Random Forest method were 86.03%, 86.32%, 85.37%, 93.18%, 72.92% and 85.84% respectively. According to the obtained results, the observations whose results were obtained, the values of Follicle (No) L. and Follicle (No) R. in different value ranges were positively correlated with the absence of PCOS. For the observations whose absence of PCOS results were obtained, the variables RBS(mg/dl), bmi_y, fsh_lh, TSH (mIU/L), Endometrium (mm) also played a role in obtaining the results. In addition, for the observations whose results were obtained, the values of Follicle No L and Follicle No R in different value ranges were also found to be positively correlated with PCOS. In addition, beta-HCG(mIU/mL), PRG(ng/mL), RBS(mg/dl), bmi_y, Endometrium (mm), fsh_lh variables also played a role in obtaining the results for PCOS. Conclusion: When the observations obtained from the results are examined, it can be said that the Follicle (No) L. and Follicle (No) R. variables are the most effective variables on the presence or absence of PCOS. For different value ranges of these two variables, the result of PCOS or not varies. Based on this, it can be said that different values of Follicle (No) L. and Follicle (No) R. variables for PCOS status may be effective in determining the disease.

show abstract

Evaluating Local Interpretable Model-Agnostic Explanations on Clinical Machine Learning Classification Models

Cited by 42 publications

References 9 publications

Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualization

Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualization

Machine Learning Model Identifies Increased Operative Time and Greater BMI as Predictors for Overnight Admission After Outpatient Hip Arthroscopy

Detection of risk factors of PCOS patients with Local Interpretable Model-agnostic Explanations (LIME) Method that an explainable artificial intelligence model

Contact Info

Product

Resources

About