Causal Perception in Question-Answering Systems

Abstract

Root cause analysis is a common data analysis task. While question-answering systems enable people to easily articulate a why question (e.g., why students in Massachusetts have high ACT Math scores on average) and obtain an answer, these systems often produce questionable causal claims. To investigate how such claims might mislead users, we conducted two crowdsourced experiments to study the impact of showing different information on user perceptions of a question-answering system. We found that in a system that occasionally provided unreasonable responses, showing a scatterplot increased the plausibility of unreasonable causal claims. Also, simply warning participants that correlation is not causation seemed to lead participants to accept reasonable causal claims more cautiously. We observed a strong tendency among participants to associate correlation with causation. Yet, the warning appeared to reduce the tendency. Grounded in the findings, we propose ways to reduce the illusion of causality when using question-answering systems.

Evaluating the Effect of Enhanced Text-Visualization Integration on Combating Misinformation in Data Story

“…The degree of text intensity can bias readers' memory of graph [43] Illusions of causality The text makes incorrect causal inductions on chart information [32,35,35] Obmiting context Omitting the context needed to understand the story [14] Manipulating order Manipulate the reading order through layout, resulting in order bias [16] Arranging Obfuscation Make it difficult for readers to extract visual information through chaotic layout [13] Reading Personal bias Political attitudes, beliefs and other personal factors lead to misperception of facts [19,38] Figure 3: The production-consumption process of narrative visualization…”

“…Annotation and Linking serve for bridging the perceptual gap between the text and visualization, which induce readers to pay attention to data-related narratives. By adopting other attention-guiding methods to nudge readers reading data stories comprehensively and skeptically (e.g., explicit warning [32]), we can have solutions to "Inattention to elements related misinformation", the most important cause identified in our study for being unaware of misinformation.…”

XInsight: eXplainable Data Analysis Through The Lens of Causality

“…(2) Causal Claim Assessment. Following [22], we transform causal relations into human-comprehensible natural language assertions and ask participants to independently evaluate them. (3) Follow-up Discussion.…”

“…While valuable for data analysis, they may confuse or mislead users who demand causal explanations. According to [22], Tableau's Explain Data reports that Massachusetts' low teen pregnancy rate may explain this state's high ACT Math score. Such explanations are questionable.…”

Comparison Conundrum and the Chamber of Visualizations: An Exploration of How Language Influences Visual Design

“…Hearst et al [33] explore appropriate visualization responses to vagueness by interpreting singular and plural superlatives (e.g., "highest price" and "highest prices") and numerical graded adjectives (e.g., "higher") based on the shape of the data distributions. Law et al [44] investigated how the visual design of answers to why questions might influence user perceptions of a question-answering system. They found that users have a strong tendency to associate correlation with causation when systems do not provide clear explanations for the answers.…”