Multiscale geometric methods for data sets I: Multiscale SVD, noise and curvature

Abstract

Large data sets are often modeled as being noisy samples from probability distributions µ in R D , with D large. It has been noticed that oftentimes the support M of these probability distributions seems to be well-approximated by low-dimensional sets, perhaps even by manifolds. We shall consider sets that are locally well approximated by k-dimensional planes, with k ≪ D, with k-dimensional manifolds isometrically embedded in R D being a special case. Samples from µ are furthermore corrupted by D-dimensional noise. Certain tools from multiscale geometric measure theory and harmonic analysis seem well-suited to be adapted to the study of samples from such probability distributions, in order to yield quantitative geometric information about them. In this paper we introduce and study multiscale covariance matrices, i.e. covariances corresponding to the distribution restricted to a ball of radius r, with a fixed center and varying r, and under rather general geometric assumptions we study how their empirical, noisy counterparts behave. We prove that in the range of scales where these covariance matrices are most informative, the empirical, noisy covariances are close to their expected, noiseless counterparts. In fact, this is true as soon as the number of samples in the balls where the covariance matrices are computed is linear in the intrinsic dimension of M. As an application, we present an algorithm for estimating the intrinsic dimension of M.

Sharp asymptotic and finite-sample rates of convergence of empirical measures in Wasserstein distance

“…Our second goal is to obtain finite-sample results which hold outside the asymptotic regime. A common phenomenon in practice is for a measure to exhibit different dimensional structure at different scales; this so-called multi-scale behavior arises in a range of applications [LMR16,WDCB05,SMB98]. We show that the convergence ofμ n to µ in W p for such measures can exhibit wildly different rates as n increases.…”

Intrinsic dimension estimation for locally undersampled data

“…Principal component analysis (PCA) is the main representative of this class of algorithms. Both a global (gPCA) and a multiscale version (mPCA) of the algorithm are used [15,23]. In the former one evaluates the covariance matrix on the whole dataset X, whereas in the latter one performs the spectral analysis on local subsets X(x 0 , r c ) of X, obtained by selecting one particular point x 0 and including in the local covariance matrix only those points that lie inside a cutoff radius r c , which is then varied.…”

“…However, the method presented above is global, thus it is expected to fail on manifolds with high intrinsic curvature. As in the case of the global PCA [15,23], we can overcome this issue by providing a suitable multiscale generalization of the FCI estimator. The multiscale FCI method selects single points in the dataset and their neighbours at a fixed maximum distance r cutoff , which is then varied.…”

Connecting dots: from local covariance to empirical intrinsic geometry and locally linear embedding

“…The local covariance matrix and its relationship with the embedded tangent space of the manifold have been widely studied recently, including (but not exclusively) [23,5,28,3,12,14]. Recently, in order to systematically study the LLE algorithm, the higher order structure of the local covariance matrix was explored in [30].…”