Visualizing High Dimensional Dynamical Processes

Duque, Andrés; Wolf, Guy; Moon, Kevin R.

doi:10.1109/mlsp.2019.8918875

Cited by 8 publications

(9 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many machine learning methods depend on some measure of pairwise similarity (which is usually unsupervised) including dimensionality reduction methods [17], [18], [19], [20], [21], [22], [23], spectral clustering [24], and any method involving the kernel trick such as SVM [25] and kernel PCA [26]. Random forest proximities can be used to extend many of these problems to a supervised setting and have been used for data visualization [27], [28], [29], [30], [31], outlier detection [30], [32], [33], [34], and data imputation [35], [36], [37], [38].…”

Section: Introductionmentioning

confidence: 99%

Geometry- and Accuracy-Preserving Random Forest Proximities

Rhodes¹,

Cutler²,

Moon³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Random forests are considered one of the best out-of-the-box classification and regression algorithms due to their high level of predictive performance with relatively little tuning. Pairwise proximities can be computed from a trained random forest which measure the similarity between data points relative to the supervised task. Random forest proximities have been used in many applications including the identification of variable importance, data imputation, outlier detection, and data visualization. However, existing definitions of random forest proximities do not accurately reflect the data geometry learned by the random forest. In this paper, we introduce a novel definition of random forest proximities called Random Forest-Geometry-and Accuracy-Preserving proximities (RF-GAP). We prove that the proximity-weighted sum (regression) or majority vote (classification) using RF-GAP exactly match the out-of-bag random forest prediction, thus capturing the data geometry learned by the random forest. We empirically show that this improved geometric representation outperforms traditional random forest proximities in tasks such as data imputation and provides outlier detection and visualization results consistent with the learned data geometry.

show abstract

Section: Introductionmentioning

confidence: 99%

Geometry- and Accuracy-Preserving Random Forest Proximities

Rhodes¹,

Cutler²,

Moon³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…On the other extreme (γ = + 1), the resulting information distance yields an L 2 distance between localized diffusion energy potentials given by U t x ðÞ ¼ log p t x ðzÞ I , as discussed by Moon et al 10 . There, as well as in other work 40,41 , it has been shown that this potential distance is amenable to a low-dimensional embedding that captures and visually accentuates emergent global and local structures in the data. Therefore, the PHATE method is based on embedding potential distances directly into two-or three-dimensional coordinates via a stress-minimizing optimization procedure provided by MDS.…”

Section: Multiscale Phate Algorithm the Multiscale Phate Algorithm Is...mentioning

confidence: 73%

Multiscale PHATE identifies multimodal signatures of COVID-19

et al. 2022

Self Cite

View full text Add to dashboard Cite

As the biomedical community produces datasets that are increasingly complex and high dimensional, there is a need for more sophisticated computational tools to extract biological insights. We present Multiscale PHATE, a method that sweeps through all levels of data granularity to learn abstracted biological features directly predictive of disease outcome. Built on a coarse-graining process called diffusion condensation, Multiscale PHATE learns a data topology that can be analyzed at coarse resolutions for high-level summarizations of data and at fine resolutions for detailed representations of subsets. We apply Multiscale PHATE to a coronavirus disease 2019 (COVID-19) dataset with 54 million cells from 168 hospitalized patients and find that patients who die show CD16 hi CD66b lo neutrophil and IFN-γ + granzyme B + Th17 cell responses. We also show that population groupings from Multiscale PHATE directly fed into a classifier predict disease outcome more accurately than naive featurizations of the data. Multiscale PHATE is broadly generalizable to different data types, including flow cytometry, single-cell RNA sequencing (scRNA-seq), single-cell sequencing assay for transposase-accessible chromatin (scATAC-seq), and clinical variables.Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

show abstract

“…The potential distance, D t , is computed by log-transforming entries of P t and then computing the pairwise distances between the log-transformed columns. Other works have shown this diffusion potential is capable of mapping out noisy and complex data in biological and other contexts [1, 16, 19]. The potentially complex, nonlinear structural relationships are finally mapped to a 2- or 3-dimensional embedding, G , via metric MDS.…”

Section: Resultsmentioning

confidence: 99%

“…Output: The RF-PHATE embedding, G Train the random forest (RF) on M Generate RF-GAP [14] proximities ( P ) from RF Row-normalize P to form the initial diffusion operator P Apply damping [15] to form Perform t steps as selected using VNE to form P t β Calculate the potential distance [1, 16] D t from P t β by applying the log transform of P t β and then computing pairwise distances between columns Form the embedding G by applying MDS to D t …”

Section: Methodsmentioning

confidence: 99%

Gaining Biological Insights through Supervised Data Visualization

Rhodes,

Aumon,

Morin

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

Dimensionality reduction-based data visualization is pivotal in comprehending complex biological data. The most common methods, such as PHATE, t-SNE, and UMAP, are unsupervised and therefore reflect the dominant structure in the data, which may be independent of expert-provided labels. Here we introduce a supervised data visualization method called RF-PHATE, which integrates expert knowledge for further exploration of the data. RF-PHATE leverages random forests to capture intricate featurelabel relationships. Extracting information from the forest, RF-PHATE generates low-dimensional visualizations that highlight relevant data relationships while disregarding extraneous features. This approach scales to large datasets and applies to classification and regression. We illustrate RF-PHATE’s prowess through three case studies. In a multiple sclerosis study using longitudinal clinical and imaging data, RF-PHATE unveils a sub-group of patients with non-benign relapsing-remitting Multiple Sclerosis, demonstrating its aptitude for time-series data. In the context of Raman spectral data, RF-PHATE effectively showcases the impact of antioxidants on diesel exhaust-exposed lung cells, highlighting its proficiency in noisy environments. Furthermore, RF-PHATE aligns established geometric structures with COVID-19 patient outcomes, enriching interpretability in a hierarchical manner. RF-PHATE bridges expert insights and visualizations, promising knowledge generation. Its adaptability, scalability, and noise tolerance underscore its potential for widespread adoption.

show abstract

Visualizing High Dimensional Dynamical Processes

Cited by 8 publications

References 25 publications

Geometry- and Accuracy-Preserving Random Forest Proximities

Geometry- and Accuracy-Preserving Random Forest Proximities

Multiscale PHATE identifies multimodal signatures of COVID-19

Gaining Biological Insights through Supervised Data Visualization

Contact Info

Product

Resources

About