Model-Agnostic Interpretability with Shapley Values

Messalas, Andreas; Kanellopoulos, Yiannis; Makris, Christos

doi:10.1109/iisa.2019.8900669

Cited by 56 publications

(41 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Shapley values provide accurate explanations, because they assign each feature an importance value for a particular prediction [31]. For example, Messalas et al [32] introduced a new metric, the top similarity method, which measures the similitude of two given explanations, produced by Shapley values, in order to evaluate the model-agnostic interpretability. Additionally, proposes a destructive method for optimizing the topology of neural networks based on the Shapley value, a game theoretic solution concept which estimates the contribution of each network element to the overall performance.…”

Section: Literature Reviewsmentioning

confidence: 99%

Darknet Traffic Big-Data Analysis and Network Management for Real-Time Automating of the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework

et al. 2021

View full text Add to dashboard Cite

Attackers are perpetually modifying their tactics to avoid detection and frequently leverage legitimate credentials with trusted tools already deployed in a network environment, making it difficult for organizations to proactively identify critical security risks. Network traffic analysis products have emerged in response to attackers’ relentless innovation, offering organizations a realistic path forward for combatting creative attackers. Additionally, thanks to the widespread adoption of cloud computing, Device Operators (DevOps) processes, and the Internet of Things (IoT), maintaining effective network visibility has become a highly complex and overwhelming process. What makes network traffic analysis technology particularly meaningful is its ability to combine its core capabilities to deliver malicious intent detection. In this paper, we propose a novel darknet traffic analysis and network management framework to real-time automating the malicious intent detection process, using a weight agnostic neural networks architecture. It is an effective and accurate computational intelligent forensics tool for network traffic analysis, the demystification of malware traffic, and encrypted traffic identification in real time. Based on a weight agnostic neural networks (WANNs) methodology, we propose an automated searching neural net architecture strategy that can perform various tasks such as identifying zero-day attacks. By automating the malicious intent detection process from the darknet, the advanced proposed solution is reducing the skills and effort barrier that prevents many organizations from effectively protecting their most critical assets.

show abstract

Section: Literature Reviewsmentioning

confidence: 99%

Darknet Traffic Big-Data Analysis and Network Management for Real-Time Automating of the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework

et al. 2021

View full text Add to dashboard Cite

show abstract

“…An SHAP chart [ 21 ] is based on Shapley values [ 20 ]. In the Shapley values theory, a prediction can be explained by assuming that each feature value of the instance is a “player” in a game, where the prediction is the payout.…”

Section: Methodsmentioning

confidence: 99%

“…Feature importance in a prediction model can be measured in various ways. Local interpretable model-agnostic explanations (LIME) [ 19 ], Shapley Values [ 20 ], and SHapley Additive ExPlanations (SHAP) [ 21 ] have been suggested to explain individual predictions. Microsoft researchers published a unified framework for machine-learning interpretability [ 21 ].…”

Section: Introductionmentioning

confidence: 99%

Explainable Machine Learning Model for Glaucoma Diagnosis and Its Interpretation

Park

Cho

et al. 2021

Diagnostics

View full text Add to dashboard Cite

The aim is to develop a machine learning prediction model for the diagnosis of glaucoma and an explanation system for a specific prediction. Clinical data of the patients based on a visual field test, a retinal nerve fiber layer optical coherence tomography (RNFL OCT) test, a general examination including an intraocular pressure (IOP) measurement, and fundus photography were provided for the feature selection process. Five selected features (variables) were used to develop a machine learning prediction model. The support vector machine, C5.0, random forest, and XGboost algorithms were tested for the prediction model. The performance of the prediction models was tested with 10-fold cross-validation. Statistical charts, such as gauge, radar, and Shapley Additive Explanations (SHAP), were used to explain the prediction case. All four models achieved similarly high diagnostic performance, with accuracy values ranging from 0.903 to 0.947. The XGboost model is the best model with an accuracy of 0.947, sensitivity of 0.941, specificity of 0.950, and AUC of 0.945. Three statistical charts were established to explain the prediction based on the characteristics of the XGboost model. Higher diagnostic performance was achieved with the XGboost model. These three statistical charts can help us understand why the machine learning model produces a specific prediction result. This may be the first attempt to apply “explainable artificial intelligence” to eye disease diagnosis.

show abstract

“…Many standard machine learning algorithms such as logistic regression, decision trees, decision-rules learning, or K-nearest neighbors are examples of more interpretable algorithms, whereas random forest, gradient boosting, support vector machine, neural networks and deep learning fall into the less-or non-interpretable machine learning approaches (i.e., black-box algorithms) (Luo et al, 2019). When a black-box model produces significantly better recommendations than a more interpretable model, the scheduling DSS developer may consider integrating feedback within the system (Kayande et al, 2009), with tools such as partial dependence (PD) plots, individual conditional expectation (ICE), local interpretable model-agnostic explanation (LIME), or kernel Shapley values (SHAP) to help partially understand the scheduling recommendation and to ensure trust and transparency in the decision process of the model (Messalas et al, 2019). On the other hand, if there are no specific design needs of relying on the mentioned blackbox methods as the main model for the DSS their capacity of exploiting non-linear relationships could still be used to derive richer features, such as the ones mentioned above.…”

Section: Scheduling Modelsmentioning

confidence: 99%

Decision Support System Applications for Scheduling in Professional Team Sport. The Team's Perspective

Schelling

Fernández

Ward

et al. 2021

Front. Sports Act. Living

View full text Add to dashboard Cite

Background: Periodization implies the systematic planning of training and competition with the goal of reaching the best possible performance in the most important competition. In team sports, this consists of finding a flight-and-practice schedule that maximizes the opportunities to perform the periodized contents (e.g., trips, practices, games, and days off). This process is conducted whilst considering known constraints (e.g., competitive schedule, roster availability, weather, especial events, holidays, or emotional effect of days away). The way a scheduling decision support system (DSS) leads users to make a decision should allow for flexibility, whilst minimizing users' confusion and facilitating the understanding of the recommendation given by the scheduling decision support system. Traditional approaches to solving scheduling problems use either simulation models, analytical models, heuristic approaches or a combination of these methods. When it comes to evaluate how the scheduling DSS is performing, three overarching aspects need to be reviewed: context satisfaction, process efficiency, and output quality. Appropriate training periodization and scheduling of trips and training sessions are critical for teams to optimize training and recovery processes in order to maximize health and performance. This article presents a methodological framework for designing decision-support systems for scheduling in professional team sports.

show abstract

Model-Agnostic Interpretability with Shapley Values

Cited by 56 publications

References 11 publications

Darknet Traffic Big-Data Analysis and Network Management for Real-Time Automating of the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework

Darknet Traffic Big-Data Analysis and Network Management for Real-Time Automating of the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework

Explainable Machine Learning Model for Glaucoma Diagnosis and Its Interpretation

Decision Support System Applications for Scheduling in Professional Team Sport. The Team's Perspective

Contact Info

Product

Resources

About