Interpreting Distortions in Dimensionality Reduction by Superimposing Neighbourhood Graphs

Abstract

To perform visual data exploration, many dimensionality reduction methods have been developed. These tools allow data analysts to represent multidimensional data in a 2D or 3D space, while preserving as much relevant information as possible. Yet, they cannot preserve all structures simultaneously and they induce some unavoidable distortions. Hence, many criteria have been introduced to evaluate a map's overall quality, mostly based on the preservation of neighbourhoods. Such global indicators are currently used to compare several maps, which helps to choose the most appropriate mapping method and its hyperparameters. However, those aggregated indicators tend to hide the local repartition of distortions. Thereby, they need to be supplemented by local evaluation to ensure correct interpretation of maps.In this paper, we describe a new method, called MING, for "Map Interpretation using Neighbourhood Graphs". It offers a graphical interpretation of pairs of map quality indicators, as well as local evaluation of the distortions. This is done by displaying on the map the nearest neighbours graphs computed in the data space and in the embedding. Shared and unshared edges exhibit reliable and unreliable neighbourhood information conveyed by the mapping. By this mean, analysts may determine whether proximity (or remoteness) of points on the map faithfully represents similarity (or dissimilarity) of original data, within the meaning of a chosen map quality criteria. We apply this approach to two pairs of widespread indicators: precision/recall and trustworthiness/continuity, chosen for their wide use in the community, which will allow an easy handling by users.

Measuring and Explaining the Inter-Cluster Reliability of Multidimensional Projections

“…The decomposition is usually done using a heatmap [61], Voronoi diagram [2,24,31], or 2D point-cloud [37,38]. By contrast, MING [10] explains False and Missing Neighbors by visualizing the shared amount of the nearest neighbor graphs constructed in the original and projected space. In this work, we quantified pointwise distortions by aggregating the inter-cluster distortion of the clusters and visualized them.…”

“…For each pair of circles (A, B) centered at C A , C B , respectively, we adjusted the degree of overlap by changing C A OC B from 60°to 0°with an interval of 2.5°, where O is the origin. For each projection, we measured Steadiness and Cohesiveness (k = [80, 90, 100, 110, 120], 500 iterations), T&C (k = [5,10,15,20,15]), and MRREs (k = [5,10,15,20,15]). We used different k values and used the mean of their results as the final score for soundness.…”

EMBEDR: Distinguishing Signal from Noise in Single-Cell Omics Data

“…Several efforts to characterize the quality of DR methods have been pursued [41,46,59], which can roughly be categorized as being global [30,53,[60][61][62][63][64][65][66][67][68] or local [29,45,46,69,70] in scope, and either based on preserving distances [68], neighborhoods [30, 46, 59-61, 63, 71, 72], or topology [67,73,74], but in all cases, they attempt to summarize the extent to which a given DR algorithm preserves some aspect of the original data's structure. In surveying this literature, and considering our basic principles, we find that what is still missing is an approach that not only assesses quality quantitatively and locally [45,47,50,[70][71][72][73]75], but also statistically in that it seeks to characterize the part of the natural and expected variability in quality that is due to noise.…”

“…In traditional data analyses, statistics provides a rigorous framework with which to answer these questions, but DR methods confound the statistical distinction between signal and noise. Specifically, DR methods: generically produce distortions in their representations of data, and these distortions are inhomogeneous across a representation [30,40,[43][44][45][46][47]; are often stochastic and non-linear, meaning that the robustness and reproducibility of results is hard to assess [41]; and often require user specification of hyperparameters, where this specification is often based on heuristics rather than quantitative principles [10,[48][49][50]. Addressing these issues provides the motivation for this work, as recovering the ability to separate signal and noise in DR output is essential for their utilization in quantitative analyses.…”

EMBEDR: Distinguishing signal from noise in single-cell omics data

“…In traditional data analyses, statistics provides a rigorous framework with which to answer these questions, but DR methods confound the statistical distinction between signal and noise. Specifically, DR methods generically produce distortions in their representations of data, and these distortions are inhomogeneous across a representation; 30 , 40 , 43 , 44 , 45 , 46 , 47 are often stochastic and non-linear, meaning that the robustness and reproducibility of results is hard to assess; 41 and often require user specification of hyperparameters, where this specification is often based on heuristics rather than quantitative principles. 10 , 48 , 49 , 50 Addressing these issues provides the motivation for this work, as recovering the ability to separate signal and noise in DR output is essential for their utilization in quantitative analyses.…”