Do Explanations Help Users Detect Errors in Open-Domain QA? An Evaluation of Spoken vs. Visual Explanations

Abstract

While research on explaining predictions of open-domain QA systems (ODQA) is gaining momentum, most works do not evaluate whether these explanations improve user trust. Furthermore, many users interact with ODQA using voice-assistants, yet prior works exclusively focus on visual displays, risking (as we also show) incorrectly extrapolating the effectiveness of explanations across modalities. To better understand the effectiveness of ODQA explanations strategies in the wild, we conduct user studies that measure whether explanations help users correctly decide when to accept or reject an ODQA system's answer. Unlike prior work, we control for explanation modality, i.e., whether they are communicated to users through a spoken or visual interface, and contrast effectiveness across modalities. We show that explanations derived from retrieved evidence can outperform strong baselines across modalities but the best explanation strategy varies with the modality. We show common failure cases of current explanations, emphasize end-to-end evaluation of explanations, and caution against evaluating them in proxy modalities that differ from deployment. * Work done while at Facebook AI.

A Test for Evaluating Performance in Human-AI Systems

“…On the other hand, multiple studies do find evidence of synergistic human-computer systems. For instance, in the study with the highest ρ ratio in our study, [41] demonstrate how algorithms can improve human decision-making in open-domain question answering tasks. In their experiment, the condition in which humans work alone achieves an accuracy of 57% and the condition in which the algorithm works alone achieves an accuracy of 50%.…”

Establishing Trustworthiness: Rethinking Tasks and Model Evaluation

“…Specifically, we need to know what users do with the model output across multiple interactions (e.g., verify, fact check, revise, accept). For example, González et al (2021) investigate the connection between explanations (D2) and user trust in the context of question answering systems. In their study users are presented with explanations in different modalities and either accept (trust) or reject (don't trust) candidate answers.…”

Evaluating Deep Taylor Decomposition for Reliability Assessment in the Wild

“…Other work that addresses human-in-the-loop evaluation of interpretability for deep neural models (a) includes Gonzalez and Søgaard (2020) and González et al (2021), but both evaluate interpretability methods with lay people and on non-critical tasks, ignoring (b) and (c). Attempts to evaluate interpretability methods for experts performing critical tasks, have, to the best of our knowledge, been limited to automatic evaluation or evaluation against gold-standard human rationales.…”