Not-So-Random Features

Bullins, Brian; Zhang, Cyril; Zhang, Yi

doi:10.48550/arxiv.1710.10230

Cited by 3 publications

(6 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is not necessary to sample from the distribution ρ defined by K (Yang, Sindhwanim, and Mahoney 2014;Avron et al 2016;Chang et al 2017;Bullins, Zhang, and Zhang 2017;Liu et al 2020;Li et al 2019b). Given a strictly positive, even reference density ρ : R d → R + (which is associated with a reference kernel K(x, x ) = k(x − x ) via Bochner's theorem), and defining µ( ω) = ρ( ω)/ρ( ω), (2) and (3) become:…”

Section: Background: Random Fourier Featuresmentioning

confidence: 99%

“…Random Fourier features (RFF) (Rahimi and Recht 2006;Liu et al 2020) was originally developed to tackle the problem of computational complexity. RFF-based methods work by approximating the feature map underlying a kernel using a finite (D-) dimensional map obtained by sampling from a density function -typically, but not necessarily (Chang et al 2017;Liu et al 2020;Bullins, Zhang, and Zhang 2017), the spectral density corresponding to the kernel by Bochner's theorem. Using this map, the problem is re-cast in approximate feature space and solved in primal form, reducing the typical complexity to O(N D 3 ).…”

Section: Introductionmentioning

confidence: 99%

“…Building on this, methods have been developed that take advantage of the fact that the feature space is exposed to, in effect, tune the feature map. For example random features may instead be drawn to maximise some criteria, typically kernel alignment (Li et al 2019a;Yu et al 2015;Bullins, Zhang, and Zhang 2017;Sinha and Duchi 2016). Alternatively, Fourier kernel learning (FKL) (Bȃzȃvan, Li, and Sminchisescu 2012) directly learns feature weights during training, in effect making the density itself an optimisation parameter.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

TRF: Learning Kernels with Tuned Random Features

Shilton

Gupta

Rana

et al. 2022

AAAI

View full text Add to dashboard Cite

Random Fourier features (RFF) are a popular set of tools for constructing low-dimensional approximations of translation-invariant kernels, allowing kernel methods to be scaled to big data. Apart from their computational advantages, by working in the spectral domain random Fourier features expose the translation invariant kernel as a density function that may, in principle, be manipulated directly to tune the kernel. In this paper we propose selecting the density function from a reproducing kernel Hilbert space to allow us to search the space of all translation-invariant kernels. Our approach, which we call tuned random features (TRF), achieves this by approximating the density function as the RKHS-norm regularised least-squares best fit to an unknown ``true'' optimal density function, resulting in a RFF formulation where kernel selection is reduced to regularised risk minimisation with a novel regulariser. We derive bounds on the Rademacher complexity for our method showing that our random features approximation method converges to optimal kernel selection in the large N,D limit. Finally, we prove experimental results for a variety of real-world learning problems, demonstrating the performance of our approach compared to comparable methods.

show abstract

Section: Background: Random Fourier Featuresmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

TRF: Learning Kernels with Tuned Random Features

Shilton

Gupta

Rana

et al. 2022

AAAI

View full text Add to dashboard Cite

show abstract

“…For threshold functions of general SVM classifiers, a "weighted" version of the alignment is optimized [29,30]. The intuition here is that only the data points that constitute the support vectors should determine the kernel parameters.…”

Section: Kernel Alignment For the Fiducial Statementioning

confidence: 99%

Covariant quantum kernels for data with group structure

Glick¹,

Gujarati²,

Córcoles³

et al. 2021

Preprint

View full text Add to dashboard Cite

The use of kernel functions is a common technique to extract important features from data sets. A quantum computer can be used to estimate kernel entries as transition amplitudes of unitary circuits. It can be shown quantum kernels exist that, subject to computational hardness assumptions, can not be computed classically. It is an important challenge to find quantum kernels that provide an advantage in the classification of real world data. Here we introduce a class of quantum kernels that are related to covariant quantum measurements and can be used for data that has a group structure. The kernel is defined in terms of a single fiducial state that can be optimized by using a technique called kernel alignment. Quantum kernel alignment optimizes the kernel family to minimize the upper bound on the generalisation error for a given data set. We apply this general method to a specific learning problem we refer to as labeling cosets with error and implement the learning algorithm on 27 qubits of a superconducting processor.

show abstract

“…( 6) with a maximum error of by drawing O( −2 ) samples from p(ω) (see also [33]). This argument applies to rotationally invariant kernels as well [34].…”

Section: Introductionmentioning

confidence: 96%

The Power of One Qubit in Machine Learning

Ghobadi,

Oberoi,

Zahedinejhad

2019

Preprint

View full text Add to dashboard Cite

Kernel methods are used extensively in classical machine learning, especially in pattern recognition. Here we propose a kernel-based quantum machine learning algorithm which can be implemented on a near-term, intermediate-scale quantum device. Our method estimates classically intractable kernel functions, using a restricted quantum model known as "deterministic quantum computing with one qubit". Our work provides a framework for studying the role of quantum correlations other than quantum entanglement, for machine learning applications.

show abstract

Not-So-Random Features

Cited by 3 publications

References 16 publications

TRF: Learning Kernels with Tuned Random Features

TRF: Learning Kernels with Tuned Random Features

Covariant quantum kernels for data with group structure

The Power of One Qubit in Machine Learning

Contact Info

Product

Resources

About