Human-XAI Interaction: A Review and Design Principles for Explanation User Interfaces

Chromik, Michael; Butz, Andreas

doi:10.1007/978-3-030-85616-8_36

Cited by 45 publications

(13 citation statements)

References 97 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As with our new questions, we have developed refined definitions and examples to clarify and improve the applicability to all 49 other XAIQB questions as part of the analysis process. The full table can be found in the supplementary material or on OSF 6 . During software exploration, it became clear that the AI system was not self-explanatory, and users might have questions about its user interface.…”

Section: Resultsmentioning

confidence: 99%

“…As a first step, we wanted to understand how other researchers or practitioners were using the XAIQB. Many papers we found in our literature search only mention Liao et al [20] as part of a literature review (e.g., [2,6,15,16]) or for theoretical arguments (e.g., [7,28,34]). A few studies used the XAIQB as a theoretical basis.…”

Section: Explainable Ai Question Bankmentioning

confidence: 99%

See 1 more Smart Citation

Identifying Explanation Needs of End-users: Applying and Extending the XAI Question Bank

Sipos,

Schäfer,

Glinka

et al. 2023

Mensch Und Computer 2023

View full text Add to dashboard Cite

Explainable Artificial Intelligence (XAI) is concerned with making the decisions of AI systems interpretable to humans. Explanations are typically developed by AI experts and focus on algorithmic transparency and the inner workings of AI systems. Research has shown that such explanations do not meet the needs of users who do not have AI expertise. As a result, explanations are often ineffective in making system decisions interpretable and understandable. We aim to strengthen a socio-technical view of AI by following a Human-Centered Explainable Artificial Intelligence (HC-XAI) approach, which investigates the explanation needs of end-users (i.e., subject matter experts and lay users) in specific usage contexts. One of the most influential works in this area is the XAI Question Bank (XAIQB) by Liao et al. The authors propose a set of questions that end-users might ask when using an AI system, which in turn is intended to help developers and designers identify and address explanation needs. Although the XAIQB is widely referenced, there are few reports of its use in practice. In particular, it is unclear to what extent the XAIQB sufficiently captures the explanation needs of end-users and what potential problems exist in the practical application of the XAIQB. To explore these open questions, we used the XAIQB as the basis for analyzing 12 think-aloud software explorations with subject matter experts, i.e., art historians. We investigated the suitability of the XAIQB as a tool for identifying explanation needs in a specific usage context. Our analysis revealed a number of explanation needs that were missing from the question bank, but that emerged repeatedly as our study participants interacted with an AI system. We also found that some of the XAIQB questions were difficult to distinguish and required interpretation during use. Our contribution is an extension of the XAIQB with 11 new questions. In addition, we have expanded the descriptions of all new and existing questions to facilitate their use. We hope that this extension will enable HCI researchers and practitioners to use the XAIQB in practice and may provide a basis for future studies on the identification of explanation needs in different contexts.CCS Concepts: • Human-centered computing → HCI design and evaluation methods.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Explainable Ai Question Bankmentioning

confidence: 99%

Identifying Explanation Needs of End-users: Applying and Extending the XAI Question Bank

Sipos,

Schäfer,

Glinka

et al. 2023

Mensch Und Computer 2023

View full text Add to dashboard Cite

show abstract

“…More generally, all the metrics discussed in this paper should be tested against a user experiment. As shown by [6,8], there is a great variety of possible experimental setups depending on what should be explained, in which context, and for who the explanation is targeted.…”

Section: Discussionmentioning

confidence: 99%

Metrics for saliency map evaluation of deep learning explanation methods

Gomez¹,

Fréour²,

Mouchère³

2022

Preprint

View full text Add to dashboard Cite

Due to the black-box nature of deep learning models, there is a recent development of solutions for visual explanations of CNNs. Given the high cost of user studies, metrics are necessary to compare and evaluate these different methods. In this paper, we critically analyze the Deletion Area Under Curve (DAUC) and Insertion Area Under Curve (IAUC) metrics proposed by Petsiuk et al. (2018). These metrics were designed to evaluate the faithfulness of saliency maps generated by generic methods such as Grad-CAM or RISE. First, we show that the actual saliency score values given by the saliency map are ignored as only the ranking of the scores is taken into account. This shows that these metrics are insufficient by themselves, as the visual appearance of a saliency map can change significantly without the ranking of the scores being modified. Secondly, we argue that during the computation of DAUC and IAUC, the model is presented with images that are out of the training distribution which might lead to an unreliable behavior of the model being explained. To complement DAUC/IAUC, we propose new metrics that quantify the sparsity and the calibration of explanation methods, two previously unstudied properties. Finally, we give general remarks about the metrics studied in this paper and discuss how to evaluate them in a user study.

show abstract

“…The domain dependent XAI methods, dedicated to specific domain such as natural science or finance, cover around 12% of the examined articles. Other research perspectives: the requirements analysis for XAI applications [49][50][51][52][53][54], the social responsibility [55][56][57][58], the processed data types [59][60][61], and human-computer interaction (HCI) [62][63][64][65][66] are represented by circa 6%, 6%, 5%, and 5% of the examined articles, respectively. As illustrated in Figure 4b, we also quantitatively analyzed the distribution of the XAI application fields.…”

Section: Methodsmentioning

confidence: 99%

XAIR: A Systematic Metareview of Explainable AI (XAI) Aligned to the Software Development Process

Clement¹,

Kemmerzell²,

Abdelaal

et al. 2023

MAKE

View full text Add to dashboard Cite

Currently, explainability represents a major barrier that Artificial Intelligence (AI) is facing in regard to its practical implementation in various application domains. To combat the lack of understanding of AI-based systems, Explainable AI (XAI) aims to make black-box AI models more transparent and comprehensible for humans. Fortunately, plenty of XAI methods have been introduced to tackle the explainability problem from different perspectives. However, due to the vast search space, it is challenging for ML practitioners and data scientists to start with the development of XAI software and to optimally select the most suitable XAI methods. To tackle this challenge, we introduce XAIR, a novel systematic metareview of the most promising XAI methods and tools. XAIR differentiates itself from existing reviews by aligning its results to the five steps of the software development process, including requirement analysis, design, implementation, evaluation, and deployment. Through this mapping, we aim to create a better understanding of the individual steps of developing XAI software and to foster the creation of real-world AI applications that incorporate explainability. Finally, we conclude with highlighting new directions for future research.

show abstract

Human-XAI Interaction: A Review and Design Principles for Explanation User Interfaces

Cited by 45 publications

References 97 publications

Identifying Explanation Needs of End-users: Applying and Extending the XAI Question Bank

Identifying Explanation Needs of End-users: Applying and Extending the XAI Question Bank

Metrics for saliency map evaluation of deep learning explanation methods

XAIR: A Systematic Metareview of Explainable AI (XAI) Aligned to the Software Development Process

Contact Info

Product

Resources

About