Explainable Artificial Intelligence in CyberSecurity: A Survey

Capuano, Nicola; Fenza, Giuseppe; Loia, Vincenzo; Stanzione, Claudio

doi:10.1109/access.2022.3204171

Cited by 92 publications

(55 citation statements)

References 229 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For more details, we request the interested reader to refer to [1,10,22,32,39,53] for general surveys on explainability methods, or for specific domains refer to: medical [115], embedded systems [117], multimodal [56], time series [90], cybersecurity [23], and tabular data [94].…”

Section: Overview Of Explainability Methodsmentioning

confidence: 99%

On the Robustness of Explanations of Deep Neural Network Models: A Survey

Jyoti¹,

Ganesh²,

Gayala³

et al. 2022

Preprint

View full text Add to dashboard Cite

Explainability has been widely stated as a cornerstone of the responsible and trustworthy use of machine learning models. With the ubiquitous use of Deep Neural Network (DNN) models expanding to risk-sensitive and safety-critical domains, many methods have been proposed to explain the decisions of these models. Recent years have also seen concerted efforts that have shown how such explanations can be distorted (attacked) by minor input perturbations. While there have been many surveys that review explainability methods themselves, there has been no effort hitherto to assimilate the different methods and metrics proposed to study the robustness of explanations of DNN models. In this work, we present a comprehensive survey of methods that study, understand, attack, and defend explanations of DNN models. We also present a detailed review of different metrics used to evaluate explanation methods, as well as describe attributional attack and defense methods. We conclude with lessons and take-aways for the community towards ensuring robust explanations of DNN model predictions.

show abstract

Section: Overview Of Explainability Methodsmentioning

confidence: 99%

On the Robustness of Explanations of Deep Neural Network Models: A Survey

Jyoti¹,

Ganesh²,

Gayala³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Although machine learning approaches have been extensively investigated, the explainability of the models has been addressed to a very limited extent 29 as also suggested by recent surveys on cyber security attacks. 30,31 Our work tries to fill this gap by proposing a methodological approach to make the machine learning models implemented for detecting phishing websites explainable.…”

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Explainable machine learning for phishing feature detection

Calzarossa

Giudici

Zieni

2023

Quality & Reliability Eng

View full text Add to dashboard Cite

Phishing is a very dangerous security threat that affects individuals as well as companies and organizations. To fight the risks associated with this threat, it is important to detect phishing websites in a timely manner. Machine learning models work well for this purpose as they can predict phishing cases, using information on the underlying websites. In this paper, we contribute to the research on the detection of phishing websites by proposing an explainable machine learning model that can provide not only accurate predictions of phishing, but also explanations of which features are most likely associated with phishing websites. To this aim, we propose a novel feature selection model based on Lorenz Zonoids, the multidimensional extension of Gini coefficient. We illustrate our proposal on a real dataset that contains features of both phishing and legitimate websites.

show abstract

“…Explanations generated for security analysts should be generated in such a way that they are intelligible and easily understandable. Existing research uses several approaches like trees, formal language, attention scores, and saliency maps to visualize explanation [10]. Similarly, there is no fixed set of quantitative metrics to evaluate explanation methods for security.…”

Section: Security Concernsmentioning

confidence: 99%

SoK: Modeling Explainability in Security Monitoring for Trust, Privacy, and Interpretability

Bhusal¹,

Rastogi²

2022

Preprint

View full text Add to dashboard Cite

Trust, privacy, and interpretability have emerged as significant concerns for experts deploying deep learning models for security monitoring. Due to their back-box nature, these models cannot provide an intuitive understanding of the machine learning predictions, which are crucial in several decision-making applications, like anomaly detection. Security operations centers have a number of security monitoring tools that analyze logs and generate threat alerts which security analysts inspect. The alerts lack sufficient explanation on why it was raised or the context in which they occurred. Existing explanation methods for security also suffer from low fidelity and low stability and ignore privacy concerns. However, explanations are highly desirable; therefore, we systematize this knowledge on explanation models so they can ensure trust and privacy in security monitoring. Through our collaborative study of security operation centers, security monitoring tools, and explanation techniques, we discuss the strengths of existing methods and concerns vis-a-vis applications, such as security log analysis. We present a pipeline to design interpretable and privacypreserving system monitoring tools. Additionally, we define and propose quantitative metrics to evaluate methods in explainable security. Finally, we discuss challenges and enlist exciting research directions for explorations.

show abstract

Explainable Artificial Intelligence in CyberSecurity: A Survey

Cited by 92 publications

References 229 publications

On the Robustness of Explanations of Deep Neural Network Models: A Survey

On the Robustness of Explanations of Deep Neural Network Models: A Survey

Explainable machine learning for phishing feature detection

SoK: Modeling Explainability in Security Monitoring for Trust, Privacy, and Interpretability

Contact Info

Product

Resources

About