Manifold-adaptive dimension estimation

Abstract

Intuitively, learning should be easier when the data points lie on a low-dimensional submanifold of the input space. Recently there has been a growing interest in algorithms that aim to exploit such geometrical properties of the data. Oftentimes these algorithms require estimating the dimension of the manifold first. In this paper we propose an algorithm for dimension estimation and study its finite-sample behaviour. The algorithm estimates the dimension locally around the data points using nearest neighbor techniques and then combines these local estimates. We show that the rate of convergence of the resulting estimate is independent of the dimension of the input space and hence the algorithm is "manifold-adaptive". Thus, when the manifold supporting the data is low dimensional, the algorithm can be exponentially more efficient than its counterparts that are not exploiting this property. Our computer experiments confirm the obtained theoretical results.

Multiscale geometric methods for data sets I: Multiscale SVD, noise and curvature

“…It has been attempted to localize PCA to small neighborhoods [39,40,41,42], without much success [43], at least compared to what we may call volume-based methods [44,45,46,47,48,12,13,49,50,51,52,53,54], which we discuss at length in Section 7. These methods, roughly speaking, are based on empirical estimates of the volume of M ∩ B z (r), for z ∈ M and r > 0: such volume grows like r k when M has dimension k, and k is estimated by fitting the empirical volume estimates for different values of r. We expect such methods, at least when naively implemented, to both require a number of samples exponential in k (if O(1) samples exist in M ∩ B z (r 0 ), for some r 0 > 0, these algorithms require O(2 k ) points in M ∩ B z (2r 0 )), and to be highly sensitive to noise, which affects the density in high dimensions.…”

“…We find an average intrinsic dimension k = 2 ( Figure 13). [67] finds k between 3 and 4 (smaller values at large scales), [68] find k ∈ [3.65, 4.65], [51] find an intrinsic dimension k = 3 using either Takens, Grassberger Procaccia or the Smoothed Grassberger Procaccia estimators, [69] find k = 4 and k = 3 depending on the way the point-wise estimates are combined (average and voting, respectively), and finally [44] find k = 4.3. Finally, we consider some data-sets whose intrinsic dimension has not been previously analyzed.…”

Image recognition via two-dimensional random projection and nearest constrained subspace

“…where D m can be estimated in advance from the dataset by other algorithms [52][53][54][55][56][57]. Based on Equation (25), we argue that for effectiveness of NCSC or even other classifiers, the number of training samples of a class should not be less than the intrinsic manifold dimension D m .…”

“…These estimators can be broadly divided into two categories: eigen projection 1 More specifically, in Equations (23) methods [58,59] and geometric methods [52][53][54][55][56][57]. Eigen projection methods estimate intrinsic dimension from the eigen decomposition of the covariance matrix of the give data.…”

“…Their estimates are given as the number of eigenvalues not less than a predefined threshold. Geometric methods, including Corr.Dim (Correlation Dimension) [53,54], MLE (Maximum Likelihood Estimate) [52] and their variations [55][56][57], exploit the intrinsic geometry of the dataset and are more sophisticated than their eigen projection counterparts [52].…”