A Systematic Literature Review of Explainable Artificial Intelligence (XAI) in Software Engineering (SE)

Abstract

Artificial intelligence (AI) is the most advanced developing area for enhancing Machine Intelligence and replicating the intelligence of humans. In this regard, Machine Learning (ML) is used to develop algorithms and models that help machines learn from data and predict problems. Although ML models provide accurate predictions, they are frequently considered black boxes due to their lack of interpretability. This can undermine trust and acceptance of AI systems, particularly in critical domains requiring transparency and accountability, such as Healthcare. Explainable Artificial Intelligence (XAI) techniques, which have emerged to make ML models more transparent and interpretable, can address the lack of interpretability challenge. They shed light on how ML models make decisions and explain and justify the results. This builds trust and makes AI systems more accessible to implement in various industries. The proposed research study investigates how much XAI is used in Software Engineering (SE). It intends to present a comprehensive view of the most recent advancements and address challenges and future directions for further investigation. This Systematic Literature Review (SLR) investigates the application of XAI techniques in SE. It is based on empirical studies published between January 2020 and September 2022 to analyze the XAI’s overall illustration. We developed a search string and six research questions, each answered briefly. According to our SLR findings, 14 of the 131 research studies extracted from various databases addressed XAI techniques. Additionally, 14 research studies using XAI techniques in the Healthcare and Finance domains were chosen to compare with the findings of this literature review. These studies were chosen because the researchers frequently cited them. Following our findings, XAI approaches were mainly employed in the Software Fault Predictions (SFP) subdomain of SE, and all studies used local explanations. Python programming libraries were used for implementation, with “sci-kit” being the most widely used, followed by “caret” of the R programming language. In addition, the “LIME” tool is the most commonly used in the SFP domain for local explanations, followed by the “SHAP” tool. The findings also show that local and global explanations were used in the Healthcare and Finance domains. The most widely used Python programming library is the “sci-kit learn” library, with the “SHAP” tool being the most commonly used explanation tool in the Finance and Healthcare domains. Finally, whereas XAI in SE is new, XAI methods have been used in conjunction with traditional machine learning models. However, there is a lack of benchmark evaluation metrics in the existing literature, leading to researcher confusion and unreliable comparison standards.