XAIR: A Framework of Explainable AI in Augmented Reality

Abstract

Explainable AI (XAI) has established itself as an important component of AI-driven interactive systems. With Augmented Reality (AR) becoming more integrated in daily lives, the role of XAI also becomes essential in AR because end-users will frequently interact with intelligent services. However, it is unclear how to design effective XAI experiences for AR. We propose XAIR, a design framework that addresses when, what, and how to provide explanations of AI output in AR. The framework was based on a multi-disciplinary literature review of XAI and HCI research, a large-scale survey probing 500+ end-users' preferences for AR-based explanations, and three workshops with 12 experts collecting their insights about XAI design in AR. XAIR's utility and effectiveness was verified via a study with 10 designers and another study with 12 end-users. XAIR can provide guidelines for designers, inspiring them to identify new design opportunities and achieve effective XAI designs in AR.

Virtual health assistants: a grand challenge in health communications and behavior change

“…A range of emerging technical solutions including explainable AI, augmented reality, digital twins, closed systems (8,9), and synthetic data, offer the potential to address current limitations and increase clinician trust in VHAs. Explainable AI enhances transparency and interpretability, enabling a clearer understanding of how AI decisions are made.…”

G-VOILA: Gaze-Facilitated Information Querying in Daily Scenarios

“…With the advancement of Augmented Reality (AR) devices, researchers have integrated IR closely with physical world through Head-Mounted Displays (HMDs) [3,14,27,39,52,71,76], which is defined as reality-based information retrieval (RBIR) by [15]. RBIR adheres to the principle of "find what you need with zero query terms or less" [7], adopting a proactive strategy to impose information on certain objects in order to reduce inquiry costs [27,39,76].…”

“…Prior research has investigated the potential of utilizing smart glasses [29,67] equipped with gaze tracking to streamline information retrieval related to physical objects [3,15,26,39,52,71,74]. These approaches enhanced information retrieval by superimposing digital widgets around physical entities, allowing users to access information through interacting with the widgets in a multi-stage manner [3,14].…”