Local intrinsic dimensionality estimators based on concentration of measure

Bac, Jonathan; Zinovyev, Andrei

doi:10.1109/ijcnn48605.2020.9207096

Cited by 7 publications

(10 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Following 20 , we consider the MNIST (focusing on the training points representing digit 1: n = 6742, D = 7797 ) and the Isolet datasets ( n = 784, D = 617 ). Moreover, we consider the Isomap faces dataset ( n = 698, D = 4096 ) as in 33,43 , and the CIFAR-10 dataset as in 44 (training data, n = 50000, D = 3072).…”

Section: Comparison Of the Evolution Of Likelihood-based Id Estimates...mentioning

confidence: 99%

The generalized ratios intrinsic dimension estimator

Denti

Doimo

Laio

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Modern datasets are characterized by numerous features related by complex dependency structures. To deal with these data, dimensionality reduction techniques are essential. Many of these techniques rely on the concept of intrinsic dimension (), a measure of the complexity of the dataset. However, the estimation of this quantity is not trivial: often, the depends rather dramatically on the scale of the distances among data points. At short distances, the can be grossly overestimated due to the presence of noise, becoming smaller and approximately scale-independent only at large distances. An immediate approach to examining the scale dependence consists in decimating the dataset, which unavoidably induces non-negligible statistical errors at large scale. This article introduces a novel statistical method, , that allows estimating the as an explicit function of the scale without performing any decimation. Our approach is based on rigorous distributional results that enable the quantification of uncertainty of the estimates. Moreover, our method is simple and computationally efficient since it relies only on the distances among data points. Through simulation studies, we show that is asymptotically unbiased, provides comparable estimates to other state-of-the-art methods, and is more robust to short-scale noise than other likelihood-based approaches.

show abstract

Section: Comparison Of the Evolution Of Likelihood-based Id Estimates...mentioning

confidence: 99%

The generalized ratios intrinsic dimension estimator

Denti

Doimo

Laio

et al. 2022

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Compared to similar efforts in other languages, the package puts emphasis on estimators, quantifying various properties of high-dimensional data geometry, such as the concentration of measure. It is the only package to include ID estimation based on linear separability of data, using Fisher discriminants [4,32,50,51].…”

Section: Related Softwarementioning

confidence: 99%

Scikit-Dimension: A Python Package for Intrinsic Dimension Estimation

Bac

Mirkes

Gorban

et al. 2021

Entropy

Self Cite

View full text Add to dashboard Cite

Dealing with uncertainty in applications of machine learning to real-life data critically depends on the knowledge of intrinsic dimensionality (ID). A number of methods have been suggested for the purpose of estimating ID, but no standard package to easily apply them one by one or all at once has been implemented in Python. This technical note introduces scikit-dimension, an open-source Python package for intrinsic dimension estimation. The scikit-dimension package provides a uniform implementation of most of the known ID estimators based on the scikit-learn application programming interface to evaluate the global and local intrinsic dimension, as well as generators of synthetic toy and benchmark datasets widespread in the literature. The package is developed with tools assessing the code quality, coverage, unit testing and continuous integration. We briefly describe the package and demonstrate its use in a large-scale (more than 500 datasets) benchmarking of methods for ID estimation for real-life and synthetic data.

show abstract

“…Even for datasets with no ID variations, it is nontrivial to make a connection between global and local ID estimation cf. [4]. On the other hand, for applications like SCA, global ID estimation is challenging due to the lack of representative data points.…”

Section: Intrinsic Dimensionalitymentioning

confidence: 99%

InfoNEAT: Information Theory-based NeuroEvolution of Augmenting Topologies for Side-channel Analysis

Acharya¹,

Ganji²,

Forte³

2021

Preprint

View full text Add to dashboard Cite

Profiled side-channel analysis (SCA) leverages leakage from cryptographic implementations to extract the secret key. When combined with advanced methods in neural networks (NNs), profiled SCA can successfully attack even those crypto-cores assumed to be protected against SCA. Despite the rise in the number of studies devoted to NN-based SCA, existing methods could not systematically address the challenges involved in the NN-based SCA. A range of questions has remained unanswered, namely: how to choose a NN with an adequate size, how to tune the NN's hyperparameters, when to stop the training, and how to explain the performance of the NN model in quantitative terms, in the context of SCA. Our proposed approach, "InfoNEAT, " tackles these issues in a natural way. InfoNEAT relies on the concept of evolution of NNs (both the network architecture and parameters, so-called neuroevolution), enhanced by information-theoretic metrics to guide the evolution, halt it with a novel stopping criteria, and improve time-complexity and memory footprint. The performance of InfoNEAT is evaluated by applying it to publicly available datasets composed of real side-channel measurements. In addition to the considerable advantages regarding the automated configuration of NNs, InfoNEAT demonstrates significant improvements over other approaches including a reduction in the number of epochs and width of the NN (i.e., the number of nodes in a layer) by factors of at least 1.25 and 6.66, respectively. According to our assessment and on the basis of our results, this is indeed achieved without any deterioration in the performance of SCA compared to the state-of-the-art NN-based methods. CCS CONCEPTS• Security and privacy → Side-channel analysis and countermeasures; • Mathematics of computing → Information theory; • Computing methodologies → Ensemble methods; Neural networks;

show abstract

Local intrinsic dimensionality estimators based on concentration of measure

Cited by 7 publications

References 20 publications

The generalized ratios intrinsic dimension estimator

The generalized ratios intrinsic dimension estimator

Scikit-Dimension: A Python Package for Intrinsic Dimension Estimation

InfoNEAT: Information Theory-based NeuroEvolution of Augmenting Topologies for Side-channel Analysis

Contact Info

Product

Resources

About