Dense Fast Random Projections and Lean Walsh Transforms

Liberty, Edo; Ailon, Nir; Singer, Amit

doi:10.1007/978-3-540-85363-3_40

Cited by 27 publications

(12 citation statements)

References 25 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also, as defined here, n is a power of 2, but variants of this construction exist for other values of n.) Importantly, applying the randomized Hadamard transform, i.e., computing the product xDH for any vector x ∈ R n takes O(n log n) time (or even O(n log r) time if only r elements in the transformed vector need to be accessed). Applying such a structured random projection was first proposed in [71,72], it was first applied in the context of randomized matrix algorithms in [80,81], and there has been a great deal of research in recent years on variants of this basic structured random projection that are better in theory or in practice [73,81,82,83,84,85,1,86,87,88,89,90]. For example, one could choose Ω = DHS, where S is a random sampling matrix, as defined above, that represents the operation of uniformly sampling a small number of columns from the randomized Hadamard transform.…”

Section: Random Sampling and Random Projectionsmentioning

confidence: 99%

Randomized Algorithms for Matrices and Data

Mahoney¹

2012

Chapman &Amp; Hall/CRC Data Mining and Knowledge Discovery Series

334

373

View full text Add to dashboard Cite

Randomized algorithms for very large matrix problems have received a great deal of attention in recent years. Much of this work was motivated by problems in large-scale data analysis, largely since matrices are popular structures with which to model data drawn from a wide range of application domains, and this work was performed by individuals from many different research communities. While the most obvious benefit of randomization is that it can lead to faster algorithms, either in worst-case asymptotic theory and/or numerical implementation, there are numerous other benefits that are at least as important. For example, the use of randomization can lead to simpler algorithms that are easier to analyze or reason about when applied in counterintuitive settings; it can lead to algorithms with more interpretable output, which is of interest in applications where analyst time rather than just computational time is of interest; it can lead implicitly to regularization and more robust output; and randomized algorithms can often be organized to exploit modern computational architectures better than classical numerical methods.This monograph will provide a detailed overview of recent work on the theory of randomized matrix algorithms as well as the application of those ideas to the solution of practical problems in large-scale data analysis. Throughout this review, an emphasis will be placed on a few simple core ideas that underlie not only recent theoretical advances but also the usefulness of these tools in large-scale data applications. Crucial in this context is the connection with the concept of statistical leverage. This concept has long been used in statistical regression diagnostics to identify outliers; and it has recently proved crucial in the development of improved worst-case matrix algorithms that are also amenable to high-quality numerical implementation and that are useful to domain scientists. This connection arises naturally when one explicitly decouples the effect of randomization in these matrix algorithms from the underlying linear algebraic structure. This decoupling also permits much finer control in the application of randomization, as well as the easier exploitation of domain knowledge.Most of the review will focus on random sampling algorithms and random projection algorithms for versions of the linear least-squares problem and the low-rank matrix approximation problem. These two problems are fundamental in theory and ubiquitous in practice. Randomized methods solve these problems by constructing and operating on a randomized sketch of the input matrix Afor random sampling methods, the sketch consists of a small number of carefully-sampled and rescaled columns/rows of A, while for random projection methods, the sketch consists of a small number of linear combinations of the columns/rows of A. Depending on the specifics of the situation, when compared with the best previously-existing deterministic algorithms, the resulting randomized algorithms have worst-case running time that is asympt...

show abstract

Section: Random Sampling and Random Projectionsmentioning

confidence: 99%

Randomized Algorithms for Matrices and Data

Mahoney¹

2012

Chapman &Amp; Hall/CRC Data Mining and Knowledge Discovery Series

334

373

View full text Add to dashboard Cite

show abstract

“…matrices where it takes at most o(n log n) time to implement the multiplication. Please see the constructions in [1,10,16,19,20] as well as the more recent papers [2,23] for further details on related and improved constructions.…”

Section: Introductionmentioning

confidence: 99%

Isometric sketching of any set via the Restricted Isometry Property

Oymak

Recht

Soltanolkotabi

2018

Information and Inference: A Journal of the IMA

View full text Add to dashboard Cite

In this paper we show that for the purposes of dimensionality reduction certain class of structured random matrices behave similarly to random Gaussian matrices. This class includes several matrices for which matrix-vector multiply can be computed in log-linear time, providing efficient dimensionality reduction of general sets. In particular, we show that using such matrices any set from high dimensions can be embedded into lower dimensions with near optimal distortion. We obtain our results by connecting dimensionality reduction of any set to dimensionality reduction of sparse vectors via a chaining argument.for all i = 1,

show abstract

“…Before getting into details of the proof, we first introduce the following two lemmas from [16] and [27]:…”

Section: Preserving Inner Productmentioning

confidence: 99%

Hierarchical Feature Hashing for Fast Dimensionality Reduction

Zhao

Xing

2014

2014 IEEE Conference on Computer Vision and Pattern Recognition

View full text Add to dashboard Cite

Curse of dimensionality is a practical and challenging problem in image categorization, especially in cases with a large number of classes. Multi-class classification encounters severe computational and storage problems when dealing with these large scale tasks. In this paper, we propose hierarchical feature hashing to effectively reduce dimensionality of parameter space without sacrificing classification accuracy, and at the same time exploit information in semantic taxonomy among categories. We provide detailed theoretical analysis on our proposed hashing method. Moreover, experimental results on object recognition and scene classification further demonstrate the effectiveness of hierarchical feature hashing.

show abstract

Dense Fast Random Projections and Lean Walsh Transforms

Cited by 27 publications

References 25 publications

Randomized Algorithms for Matrices and Data

Randomized Algorithms for Matrices and Data

Isometric sketching of any set via the Restricted Isometry Property

Hierarchical Feature Hashing for Fast Dimensionality Reduction

Contact Info

Product

Resources

About