EMBEDR: Distinguishing signal from noise in single-cell omics data

Johnson, E. M.; Kath, William L.; Mani, Madhav

doi:10.1016/j.patter.2022.100443

Cited by 16 publications

(27 citation statements)

References 82 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has already been shown that this continuous spectrum problem arises quite naturally in different practical applications: in the study of neural activity data [41][42][43], in biology [44,45], in particular with the study of single-cell data [46,47], in genetic data [48], and in financial data [49,50]. Moreover, as this problem is related to the PCA, one might expect even more applications in the future.…”

Section: Related Workmentioning

confidence: 99%

Signal Detection in Nearly Continuous Spectra and ℤ2-Symmetry Breaking

2022

View full text Add to dashboard Cite

The large scale behavior of systems having a large number of interacting degrees of freedom is suitably described using the renormalization group from non-Gaussian distributions. Renormalization group techniques used in physics are then expected to provide a complementary point of view on standard methods used in data science, especially for open issues. Signal detection and recognition for covariance matrices having nearly continuous spectra is currently an open issue in data science and machine learning. Using the field theoretical embedding introduced in Entropy, 23(9), 1132 to reproduce experimental correlations, we show in this paper that the presence of a signal may be characterized by a phase transition with Z2-symmetry breaking. For our investigations, we use the nonperturbative renormalization group formalism, using a local potential approximation to construct an approximate solution of the flow. Moreover, we focus on the nearly continuous signal build as a perturbation of the Marchenko-Pastur law with many discrete spikes.

show abstract

Section: Related Workmentioning

confidence: 99%

Signal Detection in Nearly Continuous Spectra and ℤ2-Symmetry Breaking

2022

View full text Add to dashboard Cite

show abstract

“…Despite the popularity of DR methods for visualizing high-dimensional data, these methods are prone to distortions and heterogeneity in the quality of the low-dimensional visualization [6,[10][11][12][13][14]. As such, naive use of DR methods to validate, confirm, or inform research findings and directions can be susceptible to misinterpretation due to these distortions.…”

Section: Introductionmentioning

confidence: 99%

Dynamic visualization of high-dimensional data

Sun

Zou

2022

Preprint

View full text Add to dashboard Cite

Dimensionality reduction (DR) is commonly used to project highdimensional data into lower dimensions for visualization, which could then generate new insights and hypotheses. However, DR algorithms necessarily introduce distortions in the visualization and cannot faithfully represent all relations in the data. As such, there is a need for methods to assess the reliability of DR visualizations. Here we present DynamicViz, a framework for generating dynamic visualizations that capture the sensitivity of DR visualizations to perturbations in the data. DynamicVic can be applied to all commonly used DR methods. We show the utility of dynamic visualizations in diagnosing common interpretative pitfalls of static visualizations and extending existing single-cell analyses. We introduce the variance score to quantify the dynamic variability of observations in these visualizations. The variance score characterizes natural variability in the data and can be used to optimize DR algorithm implementations. We have made DynamicViz freely available to assist with the evaluation of DR visualizations.

show abstract

“…6,7 However, choosing an optimal DR method for a given dataset and analysis remains an open question. In this issue of Patterns, Johnsona et al 8 tackle this problem by developing a quantitative quality assessment scheme: empirical marginal resampling better evaluates dimensionality reduction (EM-BEDR). EMBEDR distinguishes those structures in the reduced dimension embedding consistent with those in the original high-dimensional data versus those attributable to noise, allowing users to determine which DR representation captures the structure of the original data most accurately.…”

mentioning

confidence: 99%

“…The key to EMBEDR's evaluation is the introduction of a quality statistic termed the empirical embedding statistic, which compares cell-to-cell distance distribu-tions between the original data and its reduced dimension embedding. 8 The quality statistic is generated for each DR method and compared to the distribution of quality statistics calculated on null datasets generated via marginal resampling. An empirical hypothesis test is performed comparing the sample cell's quality to the null quality distribution, with p values calculated as the probability that the observed data yield a lower-quality embedding compared to the null datasets.…”

mentioning

confidence: 99%

“…Johnson et al emphasize that the globally optimal embedding does not necessarily mean that the quality in each local cell type is ideal and that performing local optimization may facilitate identification of rare cell types. 8 As interest in scRNA-seq technologies grows, datasets are increasing in size and complexity. DR will continue to be a key step for visualizing and analyzing ll OPEN ACCESS single-cell RNA data, and identifying an optimal DR method remains a high priority.…”

mentioning

confidence: 99%

See 1 more Smart Citation