Random Laplace Feature Maps for Semigroup Kernels on Histograms

Abstract

With the goal of accelerating the training and testing complexity of nonlinear kernel methods, several recent papers have proposed explicit embeddings of the input data into low-dimensional feature spaces, where fast linear methods can instead be used to generate approximate solutions. Analogous to random Fourier feature maps to approximate shift-invariant kernels, such as the Gaussian kernel, on R d , we develop a new randomized technique called random Laplace features, to approximate a family of kernel functions adapted to the semigroup structure of R d + . This is the natural algebraic structure on the set of histograms and other non-negative data representations. We provide theoretical results on the uniform convergence of random Laplace features. Empirical analyses on image classification and surveillance event detection tasks demonstrate the attractiveness of using random Laplace features relative to several other feature maps proposed in the literature.

Gauss-Legendre Features for Gaussian Process Regression

“…The previous two examples were of shift-invariant kernels, and the feature mapping ϕ was based on Bochner's theorem. In this section, we demonstrate the application of our theory to a different type of kernels: semigroup kernels [43]. These type of kernels require a slight modification of our setup, which we briefly describe below, but adjusting theory itself is technical and we omit it.…”

RFN: A Random-Feature Based Newton Method for Empirical Risk Minimization in Reproducing Kernel Hilbert Spaces

“…where {ω s } m s=1 are independent samples from the density τ (ω) and are called random features. There exist many kernels taking the form (5), including shift-invariant kernels (15) and dot product (e.g., polynomial) kernels (26) (see Table 1 in (27) for an exhaustive list). Gaussian kernel, for example, can be…”

Generalization Properties of Learning with Random Features

“…A lot of works have been devoted to develop random feature maps in the the setting introduced above, or slight variations (see for example [13,38,20,25,39,40,41,42,43,44] and references therein). In the rest of the section, we give several examples.…”

“…Random Laplace Feature Maps for Semigroup invariant Kernels [42] The considered input space is X = [0, ∞) d and the considered kernels are of the form K(x, z) = v(x + z), ∀x, z ∈ X, and v : X → R is a function that is positive semidefinite. By Berg's theorem, it is equivalent to the fact that v, the Laplace transform of v is such that v(ω) ≥ 0, for all ω ∈ X and that X v(ω)dω = V < ∞.…”