On variants of the Johnson–Lindenstrauss lemma

Abstract. The Johnson-Lindenstrauss lemma is a fundamental result in probability with several applications in the design and analysis of algorithms. Constructions of linear embeddings satisfying the JohnsonLindenstrauss property necessarily involve randomness and much attention has been given to obtain explicit constructions minimizing the number of random bits used. In this work we give explicit constructions with an almost optimal use of randomness: For 0 < ε, δ < 1/2, we obtain explicit generators G : {0, 1} r → R s×d for s = O(log(1/δ)/ε 2 ) such that for all d-dimensional vectors w of Euclidean norm 1,with seed-length r = O log d + log(1/δ) · log log(1/δ) ε . In particular, for δ = 1/ poly(d) and fixed ε > 0, we obtain seed-length O((log d)(log log d)). Previous constructions required Ω(log 2 d) random bits to obtain polynomially small error. We also give a new elementary proof of the optimality of the JL lemma showing a lower bound of Ω(log(1/δ)/ε 2 ) on the embedding dimension. Previously, Jayram and Woodruff [9] used communication complexity techniques to show a similar bound.

show abstract

“…Among its many known variants (see [4], [6], [8], [13]), we use the following version originally proven in [1], [4] 1 .…”

Section: Introductionmentioning

confidence: 99%

Almost Optimal Explicit Johnson-Lindenstrauss Families

Kane

Meka

Nelson

2011

Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques

View full text Add to dashboard Cite

show abstract

“…Here, the target dimension k is much smaller than the original dimension d. The name random projections was coined after the first construction by Johnson and Lindenstrauss in [1] who showed that such mappings exist for k ∈ O(log(1/δ)/ε 2 ). Other constructions of random projection matrices have been discovered since [2,3,4,5,6]. Their properties make random projections a key player in rank-k approximation algorithms [7,8,9,10,11,12,13,14], other algorithms in numerical linear algebra [15,16,17], compressed sensing [18,19,20], and various other applications, e.g, [21,22].…”

Section: Introductionmentioning

confidence: 99%

“…entries such as in [1,2,3,4,5,6] would guarantee the desired low distortion property for the entire space of matrix, D s . 4 In their analysis they show that Ψ = AD s is a good random projection for vectors x with bounded 4 norm.…”

Section: Introductionmentioning

confidence: 99%

Dense Fast Random Projections and Lean Walsh Transforms

Liberty

Ailon

Singer

2010

Discrete Comput Geom

View full text Add to dashboard Cite

Random projection methods give distributions over k × d matrices such that if a matrix Ψ (chosen according to the distribution) is applied to a finite set of vectors xi ∈ R d the resulting vectors Ψxi ∈ R k approximately preserve the original metric with constant probability. First, we show that any matrix (composed with a random ±1 diagonal matrix) is a good random projector for a subset of vectors in R d . Second, we describe a family of tensor product matrices which we term Lean Walsh. We show that using Lean Walsh matrices as random projections outperforms, in terms or running time, the best known current result (due to Matousek) under comparable assumptions.

show abstract

“…Roughly speaking, since the average number of nonzero entries of the matrix P is just O(log 2 N ), FJLT is a fast scheme because there is a significant reduction of the amount of computation of P. In [7], J. Matousek shown that it is possible to replace the Gaussian distribution N (0, q − 1) by Bernoulli (±1) distribution without incurring the dimensionality penalty, further speeding up the computation. Then, in [8], D. Ailon et al showed a simpler variant of FJLT by replacing a sparse random matrix P by a deterministic 4-wise independent code matrix (e.g.…”

Section: Introductionmentioning

confidence: 99%

Fast and efficient dimensionality reduction using Structurally Random Matrices

Thong

Liu²,

Chen

et al. 2009

2009 IEEE International Conference on Acoustics, Speech and Signal Processing

View full text Add to dashboard Cite

Structurally Random Matrices (SRM) are first proposed in [1] as fast and highly efficient measurement operators for large scale compressed sensing applications. Motivated by the bridge between compressed sensing and the Johnson-Lindenstrauss lemma [2] , this paper introduces a related application of SRMs regarding to realizing a fast and highly efficient embedding. In particular, it shows that a SRM is also a promising dimensionality reduction transform that preserves all pairwise distances of high dimensional vectors within an arbitrarily small factor , provided that the projection dimension is on the order of O( −2 log 3 N ), where N denotes the number of ddimensional vectors. In other words, SRM can be viewed as the suboptimal Johnson-Lindenstrauss embedding that, however, owns very low computational complexity O(d log d) and highly efficient implementation that uses only O(d) random bits, making it a promising candidate for practical, large scale applications where efficiency and speed of computation are highly critical.

show abstract

On variants of the Johnson–Lindenstrauss lemma

Cited by 187 publications

References 14 publications

Almost Optimal Explicit Johnson-Lindenstrauss Families

Almost Optimal Explicit Johnson-Lindenstrauss Families

Dense Fast Random Projections and Lean Walsh Transforms

Fast and efficient dimensionality reduction using Structurally Random Matrices

Contact Info

Product

Resources

About