Who needs to know what, when?: Broadening the Explainable AI (XAI) Design Space by Looking at Explanations Across the AI Lifecycle

Dhanorkar, Shipi; Wolf, Christine T.; Qian, Kun; Xu, Anbang; Popa, Lucian; Li, Yunyao

doi:10.1145/3461778.3462131

Cited by 49 publications

(28 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This contradiction implies that being able to successfully apply an explanation does not necessarily enhance a user's assessment of the XAI system. These findings support discourse from prior work on human-centered or user-centered perspectives to explainability [20,24,67]. Participants' preference towards using modes of explanation which objectively perform poorer on task performance metrics is a clear indicator that explanations need to consider the individual dispositions of the potential end-user to engender adoption.…”

Section: Qualitative Insights and Discussionsupporting

confidence: 78%

Towards Reconciling Usability and Usefulness of Explainable AI Methodologies

Tambwekar¹,

Gombolay²

2023

Preprint

View full text Add to dashboard Cite

Interactive Artificial Intelligence (AI) agents are becoming increasingly prevalent in society. However, application of such systems without understanding them can be problematic. Black-box AI systems can lead to liability and accountability issues when they produce an incorrect decision. Explainable AI (XAI) seeks to bridge the knowledge gap, between developers and end-users, by offering insights into how an AI algorithm functions. Many modern algorithms focus on making the AI model "transparent", i.e. unveil the inherent functionality of the agent in a simpler format. However, these approaches do not cater to end-users of these systems, as users may not possess the requisite knowledge to understand these explanations in a reasonable amount of time. Therefore, to be able to develop suitable XAI methods, we need to understand the factors which influence subjective perception and objective usability. In this paper, we present a novel user-study which studies four differing XAI modalities commonly employed in prior work for explaining AI behavior, i.e. Decision Trees, Text, Programs. We study these XAI modalities in the context of explaining the actions of a self-driving car on a highway, as driving is an easily understandable real-world task and self-driving cars is a keen area of interest within the AI community. Our findings highlight internal consistency issues wherein participants perceived language explanations to be significantly more usable, however participants were better able to objectively understand the decision making process of the car through a decision tree explanation. Our work also provides further evidence of importance of integrating user-specific and situational criteria into the design of XAI systems. Our findings show that factors such as computer science experience, and watching the car succeed or fail can impact the perception and usefulness of the explanation.

show abstract

Section: Qualitative Insights and Discussionsupporting

confidence: 78%

Towards Reconciling Usability and Usefulness of Explainable AI Methodologies

Tambwekar¹,

Gombolay²

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Participants' questions reflected their desire for an actionable or utility-oriented understanding to support their end goal of optimizing code generation and programming productivity [18,50,52], such as asking the Input, Output, How and How-to XAI questions (described in Section 4) to facilitate strategies to get better outputs from the AI. This actionable understanding can also be supported by enabling follow-up actions towards their goals after seeing transparent information.…”

Section: Discussion 61 Informing Xai Approaches For Genai For Codementioning

confidence: 99%

Investigating Explainability of Generative AI for Code through Scenario-based Design

Sun

Liao²,

Müller

et al. 2022

27th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

What does it mean for a generative AI model to be explainable? The emergent discipline of explainable AI (XAI) has made great strides in helping people understand discriminative models. Less attention has been paid to generative models that produce artifacts, rather than decisions, as output. Meanwhile, generative AI (GenAI) technologies are maturing and being applied to application domains such as software engineering. Using scenario-based design and question-driven XAI design approaches, we explore users' explainability needs for GenAI in three software engineering use cases: natural language to code, code translation, and code autocompletion. We conducted 9 workshops with 43 software engineers in which real examples from state-of-the-art generative AI models were used to elicit users' explainability needs. Drawing from prior work, we also propose 4 types of XAI features for GenAI for code and gathered additional design ideas from participants. Our work explores explainability needs for GenAI for code and demonstrates how human-centered approaches can drive the technical development of XAI in novel domains. CCS CONCEPTS• Computing methodologies → Natural language generation; • Software and its engineering; • Human-centered computing → Human computer interaction (HCI); User studies;

show abstract

“…There is no one-size-fits-all explanation, however. Human-centred XAI researchers therefore study how explanations can be effectively designed, considering factors such as the application context [59][60][61], human reasoning processes [62], and end users' goals [63] or personal characteristics [61,64].…”

Section: Visualisation For Explainable Artificial Intelligencementioning

confidence: 99%

Visually Explaining Uncertain Price Predictions in Agrifood: A User-Centred Case-Study

Ooge

Verbert

2022

Agriculture

View full text Add to dashboard Cite

The rise of ‘big data’ in agrifood has increased the need for decision support systems that harvest the power of artificial intelligence. While many such systems have been proposed, their uptake is limited, for example because they often lack uncertainty representations and are rarely designed in a user-centred way. We present a prototypical visual decision support system that incorporates price prediction, uncertainty, and visual analytics techniques. We evaluated our prototype with 10 participants who are active in different parts of agrifood. Through semi-structured interviews and questionnaires, we collected quantitative and qualitative data about four metrics: usability, usefulness and needs, model understanding, and trust. Our results reveal that the first three metrics can directly and indirectly affect appropriate trust, and that perception differences exist between people with diverging experience levels in predictive modelling. Overall, this suggests that user-centred approaches are key for increasing uptake of visual decision support systems in agrifood.

show abstract

Who needs to know what, when?: Broadening the Explainable AI (XAI) Design Space by Looking at Explanations Across the AI Lifecycle

Cited by 49 publications

References 53 publications

Towards Reconciling Usability and Usefulness of Explainable AI Methodologies

Towards Reconciling Usability and Usefulness of Explainable AI Methodologies

Investigating Explainability of Generative AI for Code through Scenario-based Design

Visually Explaining Uncertain Price Predictions in Agrifood: A User-Centred Case-Study

Contact Info

Product

Resources

About