Empirically Measuring Soft Knowledge in Visualization

Kijmongkolchai, Natchaya; Abdul‐Rahman, Alfie; Chen, Min

doi:10.1111/cgf.13169

Cited by 21 publications

(29 citation statements)

References 78 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Both, the data space and the knowledge space are measured using Shannon entropy. The use of human knowledge in the reverse mapping from visualization to data was confirmed in a recent empirical study by Kijmongkolchai, Abdul-Rahman and Chen [91].…”

Section: Arguments About Patterns and Structuresmentioning

confidence: 68%

Why Visualize? Untangling a Large Network of Arguments

Streeb

El‐Assady

Keim

et al. 2021

IEEE Trans. Visual. Comput. Graphics

Self Cite

View full text Add to dashboard Cite

Fig. 1: The three major stages in constructing our network of arguments on why visualization works. First, (A) we come up with the idea to look into relations between theoretic arguments, and conceptualize the basic network structure (Sections 2 & 4). Secondly, (B) we collect additional arguments from literature and iteratively align our initial ideas with current research (Section 5). Finally, (C) we implement an interactive system for keeping track of the growing network, make it accessible online for further research, and continue iterative refinement.

show abstract

Section: Arguments About Patterns and Structuresmentioning

confidence: 68%

Why Visualize? Untangling a Large Network of Arguments

Streeb

El‐Assady

Keim

et al. 2021

IEEE Trans. Visual. Comput. Graphics

Self Cite

View full text Add to dashboard Cite

show abstract

“…It was used to frame an observation study showing that human developers usually entered a huge amount of knowledge into a machine learning model [34]. It motivated an empirical study confirming that knowledge could be detected and measured quantitatively via controlled experiments [35]. It was used to analyze the cost-benefit of different virtual reality applications [36].…”

Section: Related Workmentioning

confidence: 99%

“…As shown in some recent works, it is possible for visualization designers to estimate AC, PD, and Cost qualitatively [ 36 , 37 ] and quantitatively [ 34 , 35 ]. It is highly desirable to advance the scientific methods for quantitative estimation, towards the eventual realization of computer-assisted analysis and optimization in designing visual representations.…”

Section: Overview and Motivationmentioning

confidence: 99%

A Bounded Measure for Estimating the Benefit of Visualization (Part I): Theoretical Discourse and Conceptual Evaluation

Chen

Sbert

2022

Entropy

Self Cite

View full text Add to dashboard Cite

Information theory can be used to analyze the cost–benefit of visualization processes. However, the current measure of benefit contains an unbounded term that is neither easy to estimate nor intuitive to interpret. In this work, we propose to revise the existing cost–benefit measure by replacing the unbounded term with a bounded one. We examine a number of bounded measures that include the Jenson–Shannon divergence, its square root, and a new divergence measure formulated as part of this work. We describe the rationale for proposing a new divergence measure. In the first part of this paper, we focus on the conceptual analysis of the mathematical properties of these candidate measures. We use visualization to support the multi-criteria comparison, narrowing the search down to several options with better mathematical properties. The theoretical discourse and conceptual evaluation in this part provides the basis for further data-driven evaluation based on synthetic and experimental case studies that are reported in the second part of this paper.

show abstract

“…Their information-theoretical analysis showed that ML developers entered a huge amount of knowledge (measured in bits) into a visualization-assisted ML workflow. Kijmongkolchai et al [ 43 ] reported a study designed for detecting and measuring human knowledge used in visualization, and translated the traditional accuracy values to information-theoretic measures. They encountered an undesirable property of the Kullback–Leibler divergence in their calculations.…”

Section: Related Workmentioning

confidence: 99%

“…They encountered an undesirable property of the Kullback–Leibler divergence in their calculations. In this work, we collect empirical data to evaluate the mathematical solutions proposed to address the issue encountered in [ 43 ].…”

Section: Related Workmentioning

confidence: 99%

A Bounded Measure for Estimating the Benefit of Visualization (Part II): Case Studies and Empirical Evaluation

Chen

Abdul‐Rahman

Silver

et al. 2022

Entropy

Self Cite

View full text Add to dashboard Cite

Many visual representations, such as volume-rendered images and metro maps, feature a noticeable amount of information loss due to a variety of many-to-one mappings. At a glance, there seem to be numerous opportunities for viewers to misinterpret the data being visualized, hence, undermining the benefits of these visual representations. In practice, there is little doubt that these visual representations are useful. The recently-proposed information-theoretic measure for analyzing the cost–benefit ratio of visualization processes can explain such usefulness experienced in practice and postulate that the viewers’ knowledge can reduce the potential distortion (e.g., misinterpretation) due to information loss. This suggests that viewers’ knowledge can be estimated by comparing the potential distortion without any knowledge and the actual distortion with some knowledge. However, the existing cost–benefit measure consists of an unbounded divergence term, making the numerical measurements difficult to interpret. This is the second part of a two-part paper, which aims to improve the existing cost–benefit measure. Part I of the paper provided a theoretical discourse about the problem of unboundedness, reported a conceptual analysis of nine candidate divergence measures for resolving the problem, and eliminated three from further consideration. In this Part II, we describe two groups of case studies for evaluating the remaining six candidate measures empirically. In particular, we obtained instance data for (i) supporting the evaluation of the remaining candidate measures and (ii) demonstrating their applicability in practical scenarios for estimating the cost–benefit of visualization processes as well as the impact of human knowledge in the processes. The real world data about visualization provides practical evidence for evaluating the usability and intuitiveness of the candidate measures. The combination of the conceptual analysis in Part I and the empirical evaluation in this part allows us to select the most appropriate bounded divergence measure for improving the existing cost–benefit measure.

show abstract

Empirically Measuring Soft Knowledge in Visualization

Cited by 21 publications

References 78 publications

Why Visualize? Untangling a Large Network of Arguments

Why Visualize? Untangling a Large Network of Arguments

A Bounded Measure for Estimating the Benefit of Visualization (Part I): Theoretical Discourse and Conceptual Evaluation

A Bounded Measure for Estimating the Benefit of Visualization (Part II): Case Studies and Empirical Evaluation

Contact Info

Product

Resources

About