Let's Go to the Alien Zoo: Introducing an Experimental Framework to Study Usability of Counterfactual Explanations for Machine Learning

Abstract

To foster usefulness and accountability of machine learning (ML), it is essential to explain a model's decisions in addition to evaluating its performance. Accordingly, the field of explainable artificial intelligence (XAI) has resurfaced as a topic of active research, offering approaches to address the "how" and "why" of automated decision-making. Within this domain, counterfactual explanations (CFEs) have gained considerable traction as a psychologically grounded approach to generate posthoc explanations. To do so, CFEs highlight what changes to a model's input would have changed its prediction in a particular way. However, despite the introduction of numerous CFE approaches, their usability has yet to be thoroughly validated at the human level. Thus, to advance the field of XAI, we introduce the Alien Zoo, an engaging, web-based and game-inspired experimental framework. The Alien Zoo provides the means to evaluate usability of CFEs for gaining new knowledge from an automated system, targeting novice users in a domain-general context. As a proof of concept, we demonstrate the practical efficacy and feasibility of this approach in a user study. Our results suggest that users benefit from receiving CFEs compared to no explanation, both in terms of objective performance in the proposed iterative learning task, and subjective usability. With this work, we aim to equip research groups and practitioners with the means to easily run controlled and well-powered user studies to complement their otherwise often more technology-oriented work. Thus, in the interest of reproducible research, we provide the entire code, together with the underlying models and user data: https://github.com/ukuhl/IntroAlienZoo This work is licensed under a Creative Commons "Attribution 4.0 International" license.

Keep Your Friends Close and Your Counterfactuals Closer: Improved Learning From Closest Rather Than Plausible Counterfactual Explanations in an Abstract Setting

“…To assess this question, we rely on an interactive iterative learning task, where users repeatedly choose input values for an ML model. In a separate study, we successfully demonstrate the added benefit of providing closest CFEs compared to no explanation given this experimental framework [37].…”

“…High agreement across both groups leads us to conclude that mode of CFE presentation does not have an impact when comparing users experience closest vs. plausible CFEs, validating our hypothesis 3. Thus, the Alien Zoo design does not just elucidate benefits of providing CFEs compared to no explanation [37], it is also suitable for user evaluations of CFE methods, a yet vastly understudied aspect in the field of XAI [36].…”

“…We developed a game-like experimental design, the Alien Zoo [37]. It relies on a web-based interface to provide global access for users from diverse backgrounds, facilitating large-scale participant recruitment.…”

“…It relies on a web-based interface to provide global access for users from diverse backgrounds, facilitating large-scale participant recruitment. For a detailed account of the framework's conception and the rationale behind the corresponding design choices, see Kuhl et al [37].…”