Fast Dimension Reduction Using Rademacher Series on Dual BCH Codes

Abstract

The Fast Johnson-Lindenstrauss Transform (FJLT) was recently discovered by Ailon and Chazelle as a novel technique for performing fast dimension reduction with small distortion fromachieved by naive multiplication by random dense matrices, an approach followed by several authors as a variant of the seminal result by Johnson and Lindenstrauss (JL) from the mid 80's. In this work we show how to significantly improve the running time, for any arbitrary small fixed δ. This beats the better of FJLT and JL. Our analysis uses a powerful measure concentration bound due to Talagrand applied to Rademacher series in Banach spaces (sums of vectors in Banach spaces with random signs). The set of vectors used is a real embedding of dual BCH code vectors over GF (2). We also discuss the number of random bits used and reduction to 1 space.The connection between geometry and discrete coding theory discussed here is interesting in its own right and may be useful in other algorithmic applications as well.

Dense Fast Random Projections and Lean Walsh Transforms

“…In this work we extend and simplify the result in [25]. Here we compose any column normalized fixed matrix A with a random diagonal matrix D s and give a sufficient condition on the p norm of input vectors x for which the composition AD s is a good random projection.…”

“…This can be done using the Ailon and Liberty [25] construction serving as the random projection matrix R. R is a k ×d Johnson Lindenstrauss projection matrix which can be applied ind log(k) operations ifd = d α ≥ k 2+δ for arbitrary small δ . For the same choice of a seed as in lemma 2.2, the condition becomes d ≥ k 2+δ +2δ which can be achieved by d ≥ k 2+δ for arbitrary small δ depending on δ and δ .…”

“…Since the running time of general dimensionality reduction is at least O(d) and at most O(d log(k)) (due to [25]), matrices that can be applied within this time range are especially interesting. We claim that although Lean Walsh matrices can be applied in linear time, their ∞ requirement is weaker than that achieved by sparse projection matrices.…”

Fast and efficient dimensionality reduction using Structurally Random Matrices

“…Roughly speaking, since the average number of nonzero entries of the matrix P is just O(log 2 N ), FJLT is a fast scheme because there is a significant reduction of the amount of computation of P. In [7], J. Matousek shown that it is possible to replace the Gaussian distribution N (0, q − 1) by Bernoulli (±1) distribution without incurring the dimensionality penalty, further speeding up the computation. Then, in [8], D. Ailon et al showed a simpler variant of FJLT by replacing a sparse random matrix P by a deterministic 4-wise independent code matrix (e.g. BCH codes).…”

Dense Fast Random Projections and Lean Walsh Transforms

“…In this work we extend and simplify the result in [25]. Here we compose any column normalized fixed matrix A with a random diagonal matrix D s and give a sufficient condition on the p norm of input vectors…”