Explainable artificial intelligence for cybersecurity: a literature survey

Abstract

With the extensive application of deep learning (DL) algorithms in recent years, e.g., for detecting Android malware or vulnerable source code, artificial intelligence (AI) and machine learning (ML) are increasingly becoming essential in the development of cybersecurity solutions. However, sharing the same fundamental limitation with other DL application domains, such as computer vision (CV) and natural language processing (NLP), AI-based cybersecurity solutions are incapable of justifying the results (ranging from detection and prediction to reasoning and decision-making) and making them understandable to humans. Consequently, explainable AI (XAI) has emerged as a paramount topic addressing the related challenges of making AI models explainable or interpretable to human users. It is particularly relevant in cybersecurity domain, in that XAI may allow security operators, who are overwhelmed with tens of thousands of security alerts per day (most of which are false positives), to better assess the potential threats and reduce alert fatigue. We conduct an extensive literature review on the intersection between XAI and cybersecurity. Particularly, we investigate the existing literature from two perspectives: the applications of XAI to cybersecurity (e.g., intrusion detection, malware classification), and the security of XAI (e.g., attacks on XAI pipelines, potential countermeasures). We characterize the security of XAI with several security properties that have been discussed in the literature. We also formulate open questions that are either unanswered or insufficiently addressed in the literature, and discuss future directions of research.

Accelerating SME growth in the African context: Harnessing FinTech, AI, and cybersecurity for economic prosperity

“…Additionally, the literature forecasts emerging cyber risks from the integration of AI in cybersecurity, emphasizing the need for proactive measures to address these risks (Radanliev et al, 2022). The interplay between AI and cybersecurity extends beyond FinTech, encompassing network security, computer security, and mobile security applications (Charmet et al, 2022). Furthermore, the potential of AI in improving cybersecurity capabilities is recognized, particularly in the context of national security and defense mechanisms (Vishwanath, 2023).…”

Explainable Artificial Intelligence (XAI): Concepts and Challenges in Healthcare

“…On the one hand, XAI can help to improve the safety and security of AI systems by making it easier to detect and prevent errors and malicious behavior [50]. On the other hand, XAI can also raise privacy and security concerns, as providing explanations for AI decisions may reveal sensitive information or show how to manipulate the system, for example, by reverse engineering [51].…”

Explainable Artificial Intelligence (XAI) in Healthcare

“…On the one hand, XAI can help to improve the safety and security of AI systems by making it easier to detect and prevent errors and malicious behavior [25]. On the other hand, XAI can also raise privacy and security concerns, as providing explanations for AI decisions may reveal sensitive information or show how to manipulate the system, for example by reverse engineering [26].…”