A novel intrinsic measure of data separability

Guan, Shuyue; Loew, Murray H.

doi:10.1007/s10489-022-03395-6

Cited by 7 publications

(5 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The DSI 71 for each class was computed as: where C i and C represent class i and the number of classes, respectively, and s i is the Kolmogorov-Smirnov (KS) distance for class i defined as: Here, and represent the intra-class (ICD) and between-class (BCD) distance sets for class i , respectively. The DSI serves as a normalized indicator of data separability (i.e., DSI ∈ (0,1)).…”

Section: Methodsmentioning

confidence: 99%

“…To evaluate the distinguishability of CVs, we employed two metrics: the Gini Impurity Score (GIS), which relies on local information (i.e., neighbors of each data point) to examine overlaps between data distributions, and the Distance-based Separability Index (DSI) 71 , which provides global information by assessing both the overall data distributions and the distributions of pairwise distances between data points. To implement the GIS, we overlayed a rectangular grid onto a 2D space of CVs (obtained from one of the above DR analyses).…”

Section: Measures Of Class Separationmentioning

confidence: 99%

“…A high GIS value is, thus, indicative of a substantial class overlap and, consequently, poor class separation. The DSI 71 for each class was computed as:…”

Section: Measures Of Class Separationmentioning

confidence: 99%

See 2 more Smart Citations

Automated Collective Variable Discovery for MFSD2A transporter from molecular dynamics simulations

Oh,

Rosa,

Xie

et al. 2024

Preprint

View full text Add to dashboard Cite

Biomolecules often exhibit complex free energy landscapes in which long-lived metastable states are separated by large energy barriers. Overcoming these barriers to robustly sample transitions between the metastable states with classical molecular dynamics (MD) simulations presents a challenge. To circumvent this issue, collective variable (CV)-based enhanced sampling MD approaches are often employed. Traditional CV selection relies on intuition and prior knowledge of the system. This approach introduces bias, which can lead to incomplete mechanistic insights. Thus, automated CV detection is desired to gain a deeper understanding of the system/process. Analysis of MD data with various machine learning algorithms, such as Principal Component Analysis (PCA), Support Vector Machine (SVM), and Linear Discriminant Analysis (LDA)-based approaches have been implemented for automated CV detection. However, their performance has not been systematically evaluated on structurally and mechanistically complex biological systems. Here, we applied these methods to MD simulations of the MFSD2A (Major Facilitator Superfamily Domain 2A) lysolipid transporter in multiple functionally relevant metastable states with the goal of identifying optimal CVs that would structurally discriminate these states. Specific emphasis was on the automated detection and interpretive power of LDA-based CVs. We found that LDA methods, which included a novel gradient descent-based multiclass harmonic variant, termed GDHLDA, we developed here, outperform PCA in class separation, exhibiting remarkable consistency in extracting CVs critical for distinguishing metastable states. Furthermore, the identified CVs included features previously associated with conformational transitions in MFSD2A. Specifically, conformational shifts in transmembrane helix 7 and in residue Y294 on this helix emerged as critical features discriminating the metastable states in MFSD2A. This highlights the effectiveness of LDA-based approaches in automatically extracting from MD trajectories CVs of functional relevance that can be used to drive biased MD simulations to efficiently sample conformational transitions in the molecular system.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Measures Of Class Separationmentioning

confidence: 99%

See 1 more Smart Citation

Automated Collective Variable Discovery for MFSD2A transporter from molecular dynamics simulations

Oh,

Rosa,

Xie

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…(Wasserman, 2018). In contrast, measures of data separability such as the distance-based separability index (Guan and Loew, 2021) quantify the separability of datasets in a single scalar value. However, both approaches only contribute to the first aspect of inferring the topological structure, i.e.…”

Section: Scope Of the Studymentioning

confidence: 99%

Enhancing cluster analysis via topological manifold learning

Herrmann¹,

Kazempour²,

Scheipl³

et al. 2022

Preprint

View full text Add to dashboard Cite

We discuss topological aspects of cluster analysis and show that inferring the topological structure of a dataset before clustering it can considerably enhance cluster detection: theoretical arguments and empirical evidence show that clustering embedding vectors, representing the structure of a data manifold instead of the observed feature vectors themselves, is highly beneficial. To demonstrate, we combine manifold learning method UMAP for inferring the topological structure with density-based clustering method DBSCAN. Synthetic and real data results show that this both simplifies and improves clustering in a diverse set of low-and high-dimensional problems including clusters of varying density and/or entangled shapes. Our approach simplifies clustering because topological pre-processing consistently reduces parameter sensitivity of DBSCAN. Clustering the resulting embeddings with DBSCAN can then even outperform complex methods such as SPEC-TACL and ClusterGAN. Finally, our investigation suggests that the crucial issue in clustering does not appear to be the nominal dimension of the data or how many irrelevant features it contains, but rather how separable the clusters are in the ambient observation space they

show abstract

“…The provided degree of formality is exhausted in the definition of distributional and structural outliers in precise mathematical terms and thus serves the need for precise terminology but does not overload the work with more formalism than we think necessary to contribute to the overall scope of the study. Readers interested in more rigorous mathematical approaches to infer structures in data may, for example, consult Mordohai and Medioni (2010) for dimensionality estimation and manifold learning based on tensor voting, Niyogi et al (2011) for a topological perspective on unsupervised learning, Guan and Loew (2021) of a distance‐based measure of class separability, and Kandanaarachchi and Hyndman (2020) for outlier detection in tabular data based on dimensionality reduction.…”

Section: Introductionmentioning

confidence: 99%

A geometric framework for outlier detection in high‐dimensional data

Herrmann

Pfisterer

Scheipl

2023

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Outlier or anomaly detection is an important task in data analysis. We discuss the problem from a geometrical perspective and provide a framework which exploits the metric structure of a data set. Our approach rests on the manifold assumption, that is, that the observed, nominally high‐dimensional data lie on a much lower dimensional manifold and that this intrinsic structure can be inferred with manifold learning methods. We show that exploiting this structure significantly improves the detection of outlying observations in high dimensional data. We also suggest a novel, mathematically precise and widely applicable distinction between distributional and structural outliers based on the geometry and topology of the data manifold that clarifies conceptual ambiguities prevalent throughout the literature. Our experiments focus on functional data as one class of structured high‐dimensional data, but the framework we propose is completely general and we include image and graph data applications. Our results show that the outlier structure of high‐dimensional and non‐tabular data can be detected and visualized using manifold learning methods and quantified using standard outlier scoring methods applied to the manifold embedding vectors. This article is categorized under: Technologies > Structure Discovery and Clustering Fundamental Concepts of Data and Knowledge > Data Concepts Technologies > Visualization

show abstract

A novel intrinsic measure of data separability

Cited by 7 publications

References 30 publications

Automated Collective Variable Discovery for MFSD2A transporter from molecular dynamics simulations

Automated Collective Variable Discovery for MFSD2A transporter from molecular dynamics simulations

Enhancing cluster analysis via topological manifold learning

A geometric framework for outlier detection in high‐dimensional data

Contact Info

Product

Resources

About