Towards Design Principles for User-Centric Explainable AI in Fraud Detection

Abstract

Experts rely on fraud detection and decision support systems to analyze fraud cases, a growing problem in digital retailing and banking. With the advent of Artificial Intelligence (AI) for decision support, those experts face the black-box problem and lack trust in AI predictions for fraud. Such an issue has been tackled by employing Explainable AI (XAI) to provide experts with explained AI predictions through various explanation methods. However, fraud detection studies supported by XAI lack a user-centric perspective and discussion on how principles are deployed, both important requirements for experts to choose an appropriate explanation method. On the other hand, recent research in Information Systems (IS) and Human-Computer Interaction highlights the need for understanding user requirements to develop tailored design principles for decision support systems. In this research, we adopt a design science research methodology and IS theoretical lens to develop and evaluate design principles, which align fraud expert's tasks with explanation methods for Explainable AI decision support. We evaluate the utility of these principles using an information quality framework to interview experts in banking fraud, plus a simulation. The results show that the principles are an useful tool for designing decision support systems for fraud detection with embedded user-centric Explainable AI.

Designing a feature selection method based on explainable artificial intelligence

“…The authors evaluate and refine their artifact in three consecutive design cycles. Cirqueira et al (2021) propose a design framework for an XAI-based decision support system for fraud detection within the financial services industry. They argue that existing XAI-based fraud detection studies neglect a user-centric perspective and, therefore, integrate the concept of user-centricity in their design framework.…”

“…The current literature indicates several application domains for XAI methods with justification towards regulators and other stakeholders as one major application (Adadi and Berrada, 2018). However, existing research limits the scope of justification to the final predictions of a ML model (Chakrobartty and El-Gayar, 2021;Fernandez et al, 2022;Cirqueira et al, 2021;Zhang et al, 2020). This narrow focus leaves all prior stages within the ML pipeline-such as data collection, feature selection and model training-opaque for regulators and other stakeholders.…”

“…Current research on explainable artificial intelligence (XAI) focuses on the interpretability of opaque ML models (Arrieta et al, 2020). The primary goal is to explain the logic behind a model's predictions, which are otherwise incomprehensible for human users (see, e.g., Chakrobartty and El-Gayar, 2021;Fernandez et al, 2022;Cirqueira et al, 2021;Zhang et al, 2020). These XAI techniques mostly address predictions of already trained models; literature that addresses the explainability and justifiability of ML pre-processing, let alone feature selection, is scarce (Marcilio and Eler, 2020).…”

A nascent design theory for explainable intelligent systems

“…Apart IS-related contributions such as Förster et al ( 2020) who provide a design process for user-centric XAI systems and Herm, Wanner, et al (2022b) who introduce a taxonomy to assist user-centered XAI research, we were only able to identify a handful of DSR-based contributions that focus on user-based studies for EIS (Bunde 2021;Cirqueira et al 2021;Landwehr et al 2022;Schemmer et al 2022). and Bunde (2021) provide design principles for explainable DSS limited to detecting hate speech.…”

“…Landwehr et al (2022) derive design knowledge for image-based DSS. Further, Cirqueira et al (2021) stated design principles for XAI-based systems in fraud detection and Schemmer et al (2022) propose design principles for an XAI-based DSS at real estate appraisals.…”