Homological assessment of data representations

Barannikov, Serguei; Trofimov, Ilya; Trimbach, Ekaterina; Wang, Jun; Burnaev, Evgeny

doi:10.1117/12.2623460

Cited by 3 publications

(10 citation statements)

References 7 publications

(5 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The attention mechanism can be used to interpret the model as a mapping of similarities between words in the sequence. Different features of the attention layers can be used for a clearer and more understandable description of the model, for example the ones described in [Barannikov et al, 2022], [Kushnareva et al, 2021], and [Levy et al, 2020]. They can be used to measure the similarity between two models in an ensemble.…”

Section: Attention Layer Featuresmentioning

confidence: 99%

“…Moreover, correlations only partially reflect connections between models, as shown in [Kornblith et al, 2019]. To better measure the similarity between models, we propose the divergence-based topological data analysis as it better preserves complex interdependencies between models, introduced in [Barannikov et al, 2022].…”

Section: Models' Output Correlationmentioning

confidence: 99%

“…Specifically, we chose RT D-based topological features (Representation Topology Divergence) described in [Barannikov et al, 2022] for similarity computations. They are calculated using the algorithm via graphs and barcodes defined below.…”

Section: Topology Features-based Divergencementioning

confidence: 99%

“…RTD Finally, the RTD feature is calculated by averaging the sum of barcode intervals from several runs of R-Cross-Barcode calculation on randomized subsets of vertices for G 1 and G 2 . The full algorithm for RTD calculation can be found in [Barannikov et al, 2022]. The RTD feature characterizes the degree of topological discrepancy between the representations and produces clearer divergence measure.…”

Section: Topology Features-based Divergencementioning

confidence: 99%

See 3 more Smart Citations

Effective Training-Time Stacking for Ensembling of Deep Neural Networks

Proskura

Zaytsev

2022

Proceedings of the 2022 5th International Conference on Artificial Intelligence and Pattern Recognition

View full text Add to dashboard Cite

Ensembles are important tools for improving the performance of machine learning models. In cases related to natural language processing, ensembles boost the performance of a method due to multiple large models available in open source. However, existing approaches mostly rely on simple averaging of predictions by ensembles with equal weights for each model, ignoring differences in the quality and conformity of models. We propose to estimate weights for ensembles of NLP models using not only knowledge of their individual performance but also their similarity to each other. By adopting distance measures based on Topological Data Analysis (TDA), we improve our ensemble. The quality improves for both text classification accuracy and relevant uncertainty estimation.

show abstract

Section: Attention Layer Featuresmentioning

confidence: 99%

Section: Models' Output Correlationmentioning

confidence: 99%

Section: Topology Features-based Divergencementioning

confidence: 99%

Section: Topology Features-based Divergencementioning

confidence: 99%

See 2 more Smart Citations

Effective Training-Time Stacking for Ensembling of Deep Neural Networks

Proskura

Zaytsev

2022

Proceedings of the 2022 5th International Conference on Artificial Intelligence and Pattern Recognition

View full text Add to dashboard Cite

show abstract

“…[18][19][20] Most of the existing methods either aim to find a correlation between the embeddings 18,[21][22][23] or compare the manifolds on which the data points lie. 24,25 The latter type of method is based on the manifold hypothesis that high-dimensional real-world data points lie on a comparatively low-dimensional manifold. [26][27][28] Historically, canonical correlation analysis (CCA) was used to infer information from the cross-covariance between two sets of random variables.…”

Section: Introductionmentioning

confidence: 99%

Deciphering molecular embeddings with centered kernel alignment

Welsch,

Hirte,

Kirchmair

2024

Preprint

View full text Add to dashboard Cite

The creation of effective models is of utmost importance in various scientific and engineering domains. However, analyzing such models, especially nonlinear ones, poses significant challenges. In this context, centered kernel alignment (CKA) has emerged as a promising model analysis tool that assesses the independence between two embeddings. CKA's efficacy depends on the selection of a kernel that adequately captures the underlying properties of the compared models. We examine the properties of the linear and random forest (RF) kernel with respect to multilayer perceptrons (MLPs) and RFs to adapt the model analysis tool CKA to cheminformatics. Furthermore, we demonstrate the utility of CKA in cheminformatics in three case studies in which we (1) investigate why optimizing the radius of circular fingerprints beyond two bonds results in only minor changes in the performance of models, (2) analyze the dependence between physicochemical properties and the molecular representations induced by graph neural networks (GNNs) that use addition as readout operation, and (3) compare different graph readout operations in GNNs.

show abstract

Evaluation is key: a survey on evaluation measures for synthetic time series

Stenger,

Leppich,

Foster

et al. 2024

J Big Data

View full text Add to dashboard Cite

Synthetic data generation describes the process of learning the underlying distribution of a given real dataset in a model, which is, in turn, sampled to produce new data objects still adhering to the original distribution. This approach often finds application where circumstances limit the availability or usability of real-world datasets, for instance, in health care due to privacy concerns. While image synthesis has received much attention in the past, time series are key for many practical (e.g., industrial) applications. To date, numerous different generative models and measures to evaluate time series syntheses have been proposed. However, regarding the defining features of high-quality synthetic time series and how to quantify quality, no consensus has yet been reached among researchers. Hence, we propose a comprehensive survey on evaluation measures for time series generation to assist users in evaluating synthetic time series. For one, we provide brief descriptions or - where applicable - precise definitions. Further, we order the measures in a taxonomy and examine applicability and usage. To assist in the selection of the most appropriate measures, we provide a concise guide for fast lookup. Notably, our findings reveal a lack of a universally accepted approach for an evaluation procedure, including the selection of appropriate measures. We believe this situation hinders progress and may even erode evaluation standards to a “do as you like”-approach to synthetic data evaluation. Therefore, this survey is a preliminary step to advance the field of synthetic data evaluation.

show abstract

Homological assessment of data representations

Cited by 3 publications

References 7 publications

Effective Training-Time Stacking for Ensembling of Deep Neural Networks

Effective Training-Time Stacking for Ensembling of Deep Neural Networks

Deciphering molecular embeddings with centered kernel alignment

Evaluation is key: a survey on evaluation measures for synthetic time series

Contact Info

Product

Resources

About