An Intrinsic Dimensionality Estimator from Near-Neighbor Information

Pettis, Karl; Bailey, Thomas A.; Jain, Anil K.; Dubes, Richard C.

doi:10.1109/tpami.1979.4766873

Cited by 167 publications

(91 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Over the past decades, many characterizations of the ID of sets have been proposed [10], [11], [12], [13], [14], [15], [16], [17], [18], [19]. Projection-based learning methods such as PCA [16] can produce as a byproduct an estimate of ID.…”

Section: B Intrinsic Dimensionalitymentioning

confidence: 99%

The vulnerability of learning to adversarial perturbation increases with intrinsic dimensionality

Amsaleg

Bailey

Barbe

et al. 2017

2017 IEEE Workshop on Information Forensics and Security (WIFS)

View full text Add to dashboard Cite

Abstract-Recent research has shown that machine learning systems, including state-of-the-art deep neural networks, are vulnerable to adversarial attacks. By adding to the input object an imperceptible amount of adversarial noise, it is highly likely that the classifier can be tricked into assigning the modified object to any desired class. It has also been observed that these adversarial samples generalize well across models. A complete understanding of the nature of adversarial samples has not yet emerged. Towards this goal, we present a novel theoretical result formally linking the adversarial vulnerability of learning to the intrinsic dimensionality of the data. In particular, our investigation establishes that as the local intrinsic dimensionality (LID) increases, 1-NN classifiers become increasingly prone to being subverted. We show that in expectation, a k-nearest neighbor of a test point can be transformed into its 1-nearest neighbor by adding an amount of noise that diminishes as the LID increases. We also provide an experimental validation of the impact of LID on adversarial perturbation for both synthetic and real data, and discuss the implications of our result for general classifiers.

show abstract

Section: B Intrinsic Dimensionalitymentioning

confidence: 99%

The vulnerability of learning to adversarial perturbation increases with intrinsic dimensionality

Amsaleg

Bailey

Barbe

et al. 2017

2017 IEEE Workshop on Information Forensics and Security (WIFS)

View full text Add to dashboard Cite

show abstract

“…This assumption eliminates the possibility of using categorical variables and data that lies on a submanifold of the input space. In future work we plan to investigate a way to overcome this limitation, for example by estimating the intrinsic dimensionality of the data [18]. We will also investigate optimizations of the objective Equation (7) that go beyond the proposed method.…”

Section: Discussionmentioning

confidence: 99%

Information Theoretic Clustering Using Minimum Spanning Trees

Müller

Nowozin

Lampert

2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. In this work we propose a new information-theoretic clustering algorithm that infers cluster memberships by direct optimization of a non-parametric mutual information estimate between data distribution and cluster assignment. Although the optimization objective has a solid theoretical foundation it is hard to optimize. We propose an approximate optimization formulation that leads to an efficient algorithm with low runtime complexity. The algorithm has a single free parameter, the number of clusters to find. We demonstrate superior performance on several synthetic and real datasets.

show abstract

“…Alternative approaches to the Fukunaga-Olsen's algorithm have been proposed to estimate locally ID. Among them the Near Neighbor Algorithm [8] and the methods based on Topological Representing Networks (TRN) [9] are the most popular.…”

Section: Local Methodsmentioning

confidence: 99%

“…The weakness of Trunk's method is that it is not clear how to fix a suitable value for the threshold. An improvement (Near Neighbor Algorithm) of Trunk's method was proposed by Pettis et al [8]. Assuming that the data are locally uniformly distributed, they derive the following expression for ID:…”

Section: The Near Neighbor Algorithmmentioning

confidence: 99%

Data dimensionality estimation methods: a survey

Camastra¹

2003

Pattern Recognition

234

172

View full text Add to dashboard Cite

In this paper, data dimensionality estimation methods are reviewed. The estimation of the dimensionality of a data set is a classical problem of pattern recognition. There are some good reviews [1] in literature but they do not include more recent developments based on fractal techniques and neural autoassociators. The aim of this paper is to provide an up-to-date survey of the dimensionality estimation methods of a data set, paying special attention to the fractal-based methods.

show abstract

An Intrinsic Dimensionality Estimator from Near-Neighbor Information

Cited by 167 publications

References 20 publications

The vulnerability of learning to adversarial perturbation increases with intrinsic dimensionality

The vulnerability of learning to adversarial perturbation increases with intrinsic dimensionality

Information Theoretic Clustering Using Minimum Spanning Trees

Data dimensionality estimation methods: a survey

Contact Info

Product

Resources

About