Imagining Replications: Graphical Prediction &amp; Discrete Visualizations Improve Recall &amp; Estimation of Effect Uncertainty

When students eat breakfast very often (more than 4 times a week), their GPA is around 3.5; while when students eat breakfast not very often (less than 4 times a week), their GPA is around 3.0.Number of breakfasts eaten per week GPA Number of breakfasts eaten per week GPA Number of breakfasts eaten per week GPA When students eat breakfast very often (more than 4 times a week), their GPA is around 3.5; while when students eat breakfast not very often (less than 4 times a week), their GPA is around 3.0. Fig. 1. The same data showing the relation between eating breakfast and GPA presented via text, bar graph, line graph or scatter plot. Which depiction makes eating breakfast causing higher GPA seem more plausible to you?Abstract-Students who eat breakfast more frequently tend to have a higher grade point average. From this data, many people might confidently state that a before-school breakfast program would lead to higher grades. This is a reasoning error, because correlation does not necessarily indicate causation -X and Y can be correlated without one directly causing the other. While this error is pervasive, its prevalence might be amplified or mitigated by the way that the data is presented to a viewer. Across three crowdsourced experiments, we examined whether how simple data relations are presented would mitigate this reasoning error. The first experiment tested examples similar to the breakfast-GPA relation, varying in the plausibility of the causal link. We asked participants to rate their level of agreement that the relation was correlated, which they rated appropriately as high. However, participants also expressed high agreement with a causal interpretation of the data. Levels of support for the causal interpretation were not equally strong across visualization types: causality ratings were highest for text descriptions and bar graphs, but weaker for scatter plots. But is this effect driven by bar graphs aggregating data into two groups or by the visual encoding type? We isolated data aggregation versus visual encoding type and examined their individual effect on perceived causality. Overall, different visualization designs afford different cognitive reasoning affordances across the same data. High levels of data aggregation by graphs tend to be associated with higher perceived causality in data. Participants perceived line and dot visual encodings as more causal than bar encodings. Our results demonstrate how some visualization designs trigger stronger causal links while choosing others can help mitigate unwarranted perceptions of causality.

show abstract

“…For dot encodings, perceived causality did not differ significantly between aggregation level two (M=87. 19 33, 68.20]).…”

Section: Causation Judgment Resultsmentioning

confidence: 99%

Illusion of Causality in Visualized Data

Xiong¹,

Shapiro

Hullman³

et al. 2020

IEEE Trans. Visual. Comput. Graphics

Self Cite

View full text Add to dashboard Cite

show abstract

“…Prior work in visualization and judgment and decision making suggests that different subjective probability elicitation techniques can produce varying results, perhaps because some techniques (such as frequency framings) better align with people's internal representations of uncertainty [15,22,40]. In a second study, we assess how sensitive people's responses are to different elicitation methods, which vary in the input format for beliefs they use (i.e., continuous probability versus discrete samples).…”

Section: Developing Research Questions and Goalsmentioning

confidence: 99%

“…Sample-based Elicitation. Evidence from research on reasoning with uncertainty (e.g., on classical Bayesian reasoning tasks [12]) and uncertainty visualization [9,22,23,26,27] indicates that people are often better at thinking about uncertainty when it is framed as frequency rather than probability. One way to elicit uncertainty is through a technique that asks people to provide one sample at a time until they have exhausted their mental representation.…”

Section: Developing Elicitation Techniques and Conditionsmentioning

confidence: 99%

A Bayesian Cognition Approach to Improve Data Visualization

Kim

Walls

Krafft

et al. 2019

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Self Cite

View full text Add to dashboard Cite

People naturally bring their prior beliefs to bear on how they interpret the new information, yet few formal models exist for accounting for the influence of users' prior beliefs in interactions with data presentations like visualizations. We demonstrate a Bayesian cognitive model for understanding how people interpret visualizations in light of prior beliefs and show how this model provides a guide for improving visualization evaluation. In a first study, we show how applying a Bayesian cognition model to a simple visualization scenario indicates that people's judgments are consistent with a hypothesis that they are doing approximate Bayesian inference. In a second study, we evaluate how sensitive our observations of Bayesian behavior are to different techniques for eliciting people subjective distributions, and to different datasets. We find that people don't behave consistently with Bayesian predictions for large sample size datasets, and this difference cannot be explained by elicitation technique. In a final study, we show how normative Bayesian inference can be used as an evaluation framework for visualizations, including of uncertainty.

show abstract

“…They suggested that people using the interactive models exhibited higher accuracy in Bayesian reasoning compared to previous methods. Hullman et al (2018) suggested a novel interactive, graphical uncertainty prediction technique for communicating uncertainty in experimental results by letting users to graphically predict the possible effects from experiment replications prior to seeing the true sampling distribution. The study showed that users were able to make better predictions about replications of new experiments through this technique.…”

Section: Interactive Techniquesmentioning

confidence: 99%

Increasing Interpretability of Bayesian Probabilistic Programming Models Through Interactive Representations

Taka

Stein

Williamson

2020

Front. Comput. Sci.

View full text Add to dashboard Cite

Bayesian probabilistic modeling is supported by powerful computational tools like probabilistic programming and efficient Markov Chain Monte Carlo (MCMC) sampling. However, the results of Bayesian inference are challenging for users to interpret in tasks like decision-making under uncertainty or model refinement. Decision-makers need simultaneous insight into both the model's structure and its predictions, including uncertainty in inferred parameters. This enables better assessment of the risk overall possible outcomes compatible with observations and thus more informed decisions. To support this, we see a need for visualization tools that make probabilistic programs interpretable to reveal the interdependencies in probabilistic models and their inherent uncertainty. We propose the automatic transformation of Bayesian probabilistic models, expressed in a probabilistic programming language, into an interactive graphical representation of the model's structure at varying levels of granularity, with seamless integration of uncertainty visualization. This interactive graphical representation supports the exploration of the prior and posterior distribution of MCMC samples. The interpretability of Bayesian probabilistic programming models is enhanced through the interactive graphical representations, which provide human users with more informative, transparent, and explainable probabilistic models. We present a concrete implementation that translates probabilistic programs to interactive graphical representations and show illustrative examples for a variety of Bayesian probabilistic models.

show abstract

Imagining Replications: Graphical Prediction & Discrete Visualizations Improve Recall & Estimation of Effect Uncertainty

Cited by 53 publications

References 50 publications

Illusion of Causality in Visualized Data

Illusion of Causality in Visualized Data

A Bayesian Cognition Approach to Improve Data Visualization

Increasing Interpretability of Bayesian Probabilistic Programming Models Through Interactive Representations

Contact Info

Product

Resources

About