Dense Fast Random Projections and Lean Walsh Transforms

Liberty, Edo; Ailon, Nir; Singer, Amit

doi:10.1007/s00454-010-9309-5

Cited by 22 publications

(31 citation statements)

References 22 publications

(28 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It turns out that the preconditioning transforms also satisfy the metric preservation properties of the Johnson-Lindenstrauss lemma and can be applied quickly, but do not reduce the dimension of the data. Ailon and Liberty [18] proposed the use of non-square Hadamard-like preconditioners (the socalled lean Walsh matrices) to do the preconditioning and dimensionality reduction in the same step.…”

Section: Combining Projections With Precondition-mentioning

confidence: 99%

“…While the original proof of the JL Lemma uses measure concentration on the sphere, as well as the Brunn-Minkowski inequality [20], the lemma has been reproved and improved extensively [17,18,21,1,15,8,4,3,2,13,8,9]. Early work showed that the projections could be constructed from simple random matrices, with newer results simplifying the distributions that projections are sampled from, improving the running time of the mapping, and even derandomizing the construction.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Johnson-Lindenstrauss Transform: An Empirical Study

Venkatasubramanian¹,

Wang²

2011

2011 Proceedings of the Thirteenth Workshop on Algorithm Engineering and Experiments (ALENEX)

View full text Add to dashboard Cite

The Johnson-Lindenstrauss Lemma states that a set of n points may be embedded in a space of dimension O(log n/ε 2) while preserving all pairwise distances within a factor of (1 +) with high probability. It has inspired a number of proofs that extend the result, simplify it, and improve the efficiency of computing the resulting embedding. The lemma is a critical tool in the realm of dimensionality reduction and high dimensional approximate computational geometry. It is also employed for data mining in domains that analyze intrinsically high dimensional objects such as images and text. However, while algorithms performing the dimensionality reduction have become increasingly sophisticated, there is little understanding of the behavior of these embeddings in practice. In this paper, we present the first comprehensive study of the empirical behavior of algorithms for dimensionality reduction based on the JL Lemma. Our study answers a number of important questions about the quality of the embeddings and the performance of algorithms used to compute them. Among our key results: (i) Determining a likely range for the big-Oh constant in practice for the dimension of the target space, and demonstrating the accuracy of the predicted bounds. (ii) Finding 'best in class' algorithms over wide ranges of data size and source dimensionality, and showing that these depend heavily on parameters of the data as well its sparsity. (iii) Developing the best implementation for each method, making use of non-standard optimized code for key subroutines. (iv) Identifying critical computational bottlenecks that can spur further theoretical study of efficient algorithms.

show abstract

Section: Combining Projections With Precondition-mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

The Johnson-Lindenstrauss Transform: An Empirical Study

Venkatasubramanian¹,

Wang²

2011

2011 Proceedings of the Thirteenth Workshop on Algorithm Engineering and Experiments (ALENEX)

View full text Add to dashboard Cite

show abstract

“…As noted in Sec. II-C, the projection scheme currently requires O ( dk ), but it has been suggested effective projection can be carried out in O ( d ) operations [38]. …”

Section: Experiments and Resultsmentioning

confidence: 99%

“…The random projection can be performed by constructing a k × d random matrix; by employing this approach, projection of each point requires a matrix-vector multiply taking O ( dk ) operations. Recent theoretic work suggests that even more efficient projections are possible, with [38] proposing an algorithm to project from dimension d to dimension k with O ( d ) operations.…”

Section: Methodsmentioning

confidence: 99%

Accelerating Image Registration With the Johnson–Lindenstrauss Lemma: Application to Imaging 3-D Neural Ultrastructure With Electron Microscopy

Akselrod-Ballin

Bock

Reid

et al. 2011

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

We present a novel algorithm to accelerate feature based registration, and demonstrate the utility of the algorithm for the alignment of large transmission electron microscopy (TEM) images to create 3D images of neural ultrastructure. In contrast to the most similar algorithms, which achieve small computation times by truncated search, our algorithm uses a novel randomized projection to accelerate feature comparison and to enable global search. Further, we demonstrate robust estimation of non-rigid transformations with a novel probabilistic correspondence framework, that enables large TEM images to be rapidly brought into alignment, removing characteristic distortions of the tissue fixation and imaging process. We analyze the impact of randomized projections upon correspondence detection, and upon transformation accuracy, and demonstrate that accuracy is maintained. We provide experimental results that demonstrate significant reduction in computation time and successful alignment of TEM images.

show abstract

“…Following this proof, given the initial set of n points in R d , represented as an n × d matrix, where each feature-patch is represented by a row, let R be a d × k random matrix with R ( i, j ) = r ij ; where the independent random variables r ij are: {1 with probability 0.5, and −1 with probability 0.5}. Naively, the random projection can be performed by constructing a k × d random matrix; so that mapping each point takes O ( dk ), however recent theoretic work suggests that a projection from dimension d to dimension k can be computed with O ( d ) operations [11]. …”

Section: Methodsmentioning

confidence: 99%

Accelerating Feature Based Registration Using the Johnson-Lindenstrauss Lemma

Akselrod-Ballin

Bock

Reid

et al. 2009

Medical Image Computing and Computer-Assisted Intervention – MICCAI 2009

View full text Add to dashboard Cite

Abstract. We introduce an efficient search strategy to substantially accelerate feature based registration. Previous feature based registration algorithms often use truncated search strategies in order to achieve small computation times. Our new accelerated search strategy is based on the realization that the search for corresponding features can be dramatically accelerated by utilizing Johnson-Lindenstrauss dimension reduction. Order of magnitude calculations for the search strategy we propose here indicate that the algorithm proposed is more than a million times faster than previously utilized naive search strategies, and this advantage in speed is directly translated into an advantage in accuracy as the fast speed enables more comparisons to be made in the same amount of time. We describe the accelerated scheme together with a full complexity analysis. The registration algorithm was applied to large transmission electron microscopy (TEM) images of neural ultrastructure. Our experiments demonstrate that our algorithm enables alignment of TEM images with increased accuracy and efficiency compared to previous algorithms.

show abstract

Dense Fast Random Projections and Lean Walsh Transforms

Cited by 22 publications

References 22 publications

The Johnson-Lindenstrauss Transform: An Empirical Study

The Johnson-Lindenstrauss Transform: An Empirical Study

Accelerating Image Registration With the Johnson–Lindenstrauss Lemma: Application to Imaging 3-D Neural Ultrastructure With Electron Microscopy

Accelerating Feature Based Registration Using the Johnson-Lindenstrauss Lemma

Contact Info

Product

Resources

About