Misinformed by Visualization: What Do We Learn From Misinformative Visualizations?

Abstract

Data visualization is powerful in persuading an audience. However, when it is done poorly or maliciously, a visualization may become misleading or even deceiving. Visualizations give further strength to the dissemination of misinformation on the Internet. The visualization research community has long been aware of visualizations that misinform the audience, mostly associated with the terms "lie" and "deceptive." Still, these discussions have focused only on a handful of cases. To better understand the landscape of misleading visualizations, we open-coded over one thousand real-world visualizations that have been reported as misleading. From these examples, we discovered 74 types of issues and formed a taxonomy of misleading elements in visualizations. We found four directions that the research community can follow to widen the discussion on misleading visualizations:(1) informal fallacies in visualizations, (2) exploiting conventions and data literacy, (3) deceptive tricks in uncommon charts, and (4) understanding the designers' dilemma. This work lays the groundwork for these research directions, especially in understanding, detecting, and preventing them. CCS Concepts• Human-centered computing → Information visualization;From the taxonomy and the notes taken during the coding process, we identified four directions that would enrich the discussion on misleading visualizations: (1) informal fallacies in visualizations, (2) exploiting conventions and data literacy, (3) deceptive tricks in uncommon charts, and (4) providing design guidelines by understanding the designers' dilemma.

CALVI: Critical Thinking Assessment for Literacy in Visualizations

“…McNutt et al also compiled categories of errors that can lead to visualization mirages, such as missing records in the data curating process, overplotting, concealing uncertainty, or manipulating scales in the visualization phase [32]. More recently, Lo et al developed a taxonomy of misinformative visualizations that included errors or issues that could be present on poorly designed visualizations [30]. Previous work on visualization misinformation has informed us ways of manipulation that can result in misleading visualizations.…”

“…To compile an initial set of ways a visualization can mislead, we drew from two main prior works: categorizations from McNutt et al and Lo et al [30,32]. We reviewed each category from McNutt et al, extracted relevant categories based on main criteria such as visually detectable (we cannot test for misleaders that people cannot detect visually) and not cognitive biases (cognitive biases from readers do not ft in the defnition of misleader as they are not part of the visualization construction process 4 ), and further categorized them into higher-level or lower-level categories ( A in Figure 3).…”

“…Selecting only a subset of data to display, which can be misleading if one is Cherry Picking asked to infer something about the whole set of data. [30,32,40] Not displaying uncertainty in visualizations may misrepresent the certainty in Concealed Uncertainty the underlying data. In the case of prediction making, this can misguide the viewers to falsely overconfdent conclusions.…”

“…[25, Missing Data 32] Displaying unnormalized data in absolute quantity when normalized data in Missing Normalization relative quantity is of interest. [17,30,32,42] Overplotting Displaying too many things on a plot can obscure parts of the data. [30,32] types as columns ( G in Figure 3).…”

“…[17,30,32,42] Overplotting Displaying too many things on a plot can obscure parts of the data. [30,32] types as columns ( G in Figure 3). 6 This matrix helped us explore how misleaders can be applied across chart types to systematically construct misleading visualizations, and we refer a cell in this matrix as an item type.…”