Perception-Based Evaluation of Projection Methods for Multidimensional Data Visualization

Abstract

Abstract-Similarity-based layouts generated by multidimensional projections or other dimension reduction techniques are commonly used to visualize high-dimensional data. Many projection techniques have been recently proposed addressing different objectives and application domains. Nonetheless, very little is known about the effectiveness of the generated layouts from a user's perspective, how distinct layouts from the same data compare regarding the typical visualization tasks they support, or how domain-specific issues affect the outcome of the techniques. Learning more about projection usage is an important step towards both consolidating their role in high-dimensional data analysis and taking informed decisions when choosing techniques. This work provides a contribution towards this goal. We describe the results of an investigation on the performance of layouts generated by projection techniques as perceived by their users. We conducted a controlled user study to test against the following hypotheses: (1) projection performance is task-dependent; (2) certain projections perform better on certain types of tasks; (3) projection performance depends on the nature of the data; and (4) subjects prefer projections with good segregation capability. We generated layouts of high-dimensional data with five techniques representative of different projection approaches. As application domains we investigated image and document data. We identified eight typical tasks, three of them related to segregation capability of the projection, three related to projection precision, and two related to incurred visual cluttering. Answers to questions were compared for correctness against 'ground truth' computed directly from the data. We also looked at subject confidence and task completion times. Statistical analysis of the collected data resulted in Hypotheses 1 and 3 being confirmed, Hypothesis 2 being confirmed partially and Hypotheses 4 could not be confirmed. We discuss our findings in comparison with some numerical measures of projection layout quality. Our results offer interesting insight on the use of projection layouts in data visualization tasks and provide a departing point for further systematic investigations.

ClustMe: A Visual Quality Measure for Ranking Monochrome Scatterplots based on Cluster Patterns

“…The similarity between patterns can be measured by relative ranking using some Likert scale [TBB*10] or by manual grouping of similar scatterplot based on their thumbnail images [DBH14, PKF*16]. Cluster patterns can also be characterized by identifying their elements using a lasso selection, or the overall pattern evaluated by counting the number of clusters [EMdSP*15].…”

SolarView: Low Distortion Radial Embedding with a Focus

“…Employing dimensionality reduction techniques [11], [13], [14] is a common approach to deal with many similarities at once. Typically, the results of dimensionality reduction are visualized as 2D scatterplots, in which the distances between points merely approximate the real distances.…”

“…Typically, the results of dimensionality reduction are visualized as 2D scatterplots, in which the distances between points merely approximate the real distances. Controlled user studies have shown that users find it difficult to understand the placement of the points and how accurately the real distances have been preserved [9], and that there is a dependency between task and projection performance [13]. Users must also develop trust in the projection and representation to use the visualization confidently [10].…”

Towards Perceptual Optimization of the Visual Design of Scatterplots

“…Etemadpour et al [23] proposed perception-based evaluation of projection methods based on empirical data but not perceptual models and quality metrics as in our approach. Color optimization in scatterplots was adopted by Chen et al [24] to improve the discernibility of multiple classes when the scatterplot is overplotted.…”