Cover-based bounds on the numerical rank of Gaussian kernels

Abstract

A popular approach for analyzing high-dimensional datasets is to perform dimensionality reduction by applying non-parametric affinity kernels. Usually, it is assumed that the represented affinities are related to an underlying lowdimensional manifold from which the data is sampled. This approach works under the assumption that, due to the low-dimensionality of the underlying manifold, the kernel has a low numerical rank. Essentially, this means that the kernel can be represented by a small set of numerically-significant eigenvalues and their corresponding eigenvectors.We present an upper-bound for the numerical rank of Gaussian convolution operators, which are commonly used as kernels by spectral manifoldlearning methods. The achieved bound is based on the underlying geometry that is provided by the manifold from which the dataset is assumed to be sampled. The bound can be used to determine the number of significant eigenvalues/eigenvectors that are needed for spectral analysis purposes. Furthermore, the results in this paper provide a relation between the underlying geometry of the manifold (or dataset) and the numerical rank of its Gaussian affinities.The term cover-based bound is used because the computations of this bound are done by using a finite set of small constant-volume boxes that cover the underlying manifold (or the dataset). We present bounds for finite Gaussiankernel matrices as well as for the continuous Gaussian convolution operator. We explore and demonstrate the relations between the bounds that are achieved for finite and continuous cases. The cover-oriented methodology is also used to provide a relation between the geodesic length of a curve and the numerical rank of Gaussian kernel of datasets that are sampled from it.

Multiscale PHATE identifies multimodal signatures of COVID-19

“…Although a DM provides appealing analytic relation between spectral embedding with diffusion coordinates [37][38][39] , it often separates trajectories, pathways or clusters into independent eigenspaces. This, in turn, yields multidimensional representations that cannot be conveniently visualized (e.g., having substantially more than two or three dimensions) and cannot be directly projected into two-or three-dimensional displays that faithfully capture diffusion distances.…”

Multiscale PHATE Exploration of SARS-CoV-2 Data Reveals Multimodal Signatures of Disease

“…While several similarity kernels are used in practice to construct the diffusion operator P, a standard choice is the Gaussian affinity k ε (x, y) = exp(− x − y 2 /ε) [13,[82][83][84], in which case we denote the diffusion operator P ε , where ε determines the neighborhood radius. This kernel choice is often seen in theoretical and mathematical work due to its established properties on data sampled from locally low dimensional geometries (i.e., data manifolds) [82,85,86]. In particular, it can be verified that when the data is sampled from a Riemannian manifold, the diffusion operator P t/ε ε constructed from k ε (•, •) converges to the heat kernel on the underlying manifold as ε → 0.…”

Multiscale PHATE Exploration of SARS-CoV-2 Data Reveals Multimodal Signatures of Disease

“…While several kernels are used in practice to construct the diffusion operator P, a standard choice is the Gaussian affinity k ε (x, y) = exp(− x − y 2 /ε) [13,[74][75][76], in which case we denote the diffusion operator P ε , where ε determines the neighborhood radius. This kernel choice is often seen in theoretical and mathematical work due to its established properties on data sampled from locally low dimensional geometries (i.e., data manifolds) [74,77,78]. In particular, it can be verified that when the data is sampled from a Riemannian manifold, the diffusion operator P t/ε ε constructed from k ε (•, •) converges to the heat kernel on the underlying manifold as ε → 0.…”