An Almost Optimal Unrestricted Fast Johnson-Lindenstrauss Transform

Ailon, Nir; Liberty, Edo

doi:10.1145/2483699.2483701

Cited by 82 publications

(128 citation statements)

References 25 publications

(32 reference statements)

Supporting

Mentioning

123

Contrasting

Unclassified

Order By: Relevance

“…This was improved to O(d log d + m 2+γ ) [3] with the same m for any constant γ > 0, or to O(d log d) with m = O(ε −2 log n log 4 d) [4,40]. Thus if ε −2 log n √ d one can obtain nearly-linear O(d log d) embedding time with the same target dimension m as the original JL lemma, or one can also obtain nearlylinear time for any setting of ε, n by increasing m slightly by polylog d factors.…”

Section: Johnson-lindenstraussmentioning

confidence: 99%

See 1 more Smart Citation

Sparsity lower bounds for dimensionality reducing maps

Nelson

NguyÅn

2013

Proceedings of the Forty-Fifth Annual ACM Symposium on Theory of Computing

View full text Add to dashboard Cite

We give near-tight lower bounds for the sparsity required in several dimensionality reducing linear maps. First, consider the Johnson-Lindenstrauss (JL) lemma which states that for any set of n vectors in R d there is an A ∈ R m×d with m = O(ε −2 log n) such that mapping by A preserves the pairwise Euclidean distances up to a 1 ± ε factor 1 . We show there exists a set of n vectors such that any such A with at most s non-zero entries per column must have s = Ω(ε −1 log n/ log(1/ε)) if m < O(n/ log(1/ε)). This improves the lower bound of Ω(min{ε −2 , ε −1 log m d}) by [DasguptaKumar-Sarlós, STOC 2010], which only held against the stronger property of distributional JL, and only against a certain restricted class of distributions. Meanwhile our lower bound is against the JL lemma itself, with no restrictions. Our lower bound matches the sparse JL upper bound of [Kane-Nelson, SODA 2012] up to an O(log(1/ε)) factor.Next, we show that any m×n matrix with the k-restricted isometry property (RIP) with constant distortion must have Ω(k log(n/k)) non-zeroes per column if m = O(k log(n/k)), the optimal number of rows for RIP, and k < n/ polylog n. This improves the previous lower bound of Ω(min{k, n/m}) by [Chandar, 2010] and shows that for most k it is impossible to have a sparse RIP matrix with an optimal number of rows.Both lower bounds above also offer a tradeoff between sparsity and the number of rows.Lastly, we show that any oblivious distribution over subspace embedding matrices with 1 non-zero per column and preserving distances in a d dimensional-subspace up to a constant factor must have at least Ω(d 2 ) rows. This matches an upper bound in [Nelson-Nguyẽn, arXiv abs/1211.1002] and shows the impossibility of obtaining the best of both of constructions in that work, namely 1 non-zero per column and d · polylog d rows. We say A = (1 ± ε)B if (1 − ε)B ≤ A ≤ (1 + ε)B

show abstract

Section: Johnson-lindenstraussmentioning

confidence: 99%

“…The intuition behind all the works [2][3][4]40] to obtain O(d log d) embedding time was as follows. Picking A to be a scaled sampling matrix (where each row has a 1 in a random location) gives the correct expectation for Ax 2 2 , but the variance may be too high.…”

Section: Johnson-lindenstraussmentioning

confidence: 99%

Sparsity lower bounds for dimensionality reducing maps

Nelson

NguyÅn

2013

Proceedings of the Forty-Fifth Annual ACM Symposium on Theory of Computing

View full text Add to dashboard Cite

show abstract

“…Interesting new work of Ailon and Liberty [5] appeared during the review process of this paper. Their transformation is the composition of a random sign matrix with a random selection of a suitable number k of rows from a Fourier matrix.…”

Section: Note Added In Proofmentioning

confidence: 99%

A variant of the Johnson–Lindenstrauss lemma for circulant matrices

Vybíral

2011

Journal of Functional Analysis

View full text Add to dashboard Cite

We continue our study of the Johnson-Lindenstrauss lemma and its connection to circulant matrices started in Hinrichs and Vybíral (in press) [7]. We reduce the bound on k from k = Ω(ε −2 log 3 n) proven there to k = Ω(ε −2 log 2 n). Our technique differs essentially from the one used in Hinrichs and Vybíral (in press) [7]. We employ the discrete Fourier transform and singular value decomposition to deal with the dependency caused by the circulant structure.

show abstract

“…The literature already contains a number of papers, including [Ailon and Chazelle (2009);Liberty (2009);Nguyen et al (2009);Halko et al (2011); Ailon and Liberty (2010); Krahmer and Ward (2010)], that study the behavior of the SRHT and related dimension-reduction maps. The current treatment differs in several regards.…”

Section: Introductionmentioning

confidence: 99%

Improved Analysis of the Subsampled Randomized Hadamard Transform

Tropp

2011

Adv. Adapt. Data Anal.

244

245

View full text Add to dashboard Cite

This paper presents an improved analysis of a structured dimension-reduction map called the subsampled randomized Hadamard transform. This argument demonstrates that the map preserves the Euclidean geometry of an entire subspace of vectors. The new proof is much simpler than previous approaches, and it offers -for the first time -optimal constants in the estimate on the number of dimensions required for the embedding.Our analysis relies on two basic properties of the Walsh-Hadamard matrix: The derived matrix H is orthogonal, and its entries all have magnitude ± n −1/2 . Walsh-Hadamard matrices exist for each n = 2 p where p = 1, 2, 3, . . . . Remark 1.1. The Walsh-Hadamard matrix is the real analog of the discrete Fourier matrix. Whereas the n × n Fourier matrix displays the characters of the cyclic group of order n, the n × n Walsh-Hadamard matrix contains the characters of the additive group Z p 2 where n = 2 p . In each case, the underlying algebraic structure allows us to multiply the matrix by a vector in about n log n arithmetic operations. Note, however, that our results are purely analytic; so, they do not rely on the algebraic properties of the Walsh-Hadamard matrix. We have chosen not to work with the Fourier matrix to avoid some small complications associated with complex random variables.

show abstract

An Almost Optimal Unrestricted Fast Johnson-Lindenstrauss Transform

Cited by 82 publications

References 25 publications

Sparsity lower bounds for dimensionality reducing maps

Sparsity lower bounds for dimensionality reducing maps

A variant of the Johnson–Lindenstrauss lemma for circulant matrices

Improved Analysis of the Subsampled Randomized Hadamard Transform

Contact Info

Product

Resources

About