On the impossibility of dimension reduction in l
            <sub>1</sub>

Brinkman, Bo; Charikar, Moses

doi:10.1145/1089023.1089026

Cited by 113 publications

(156 citation statements)

References 27 publications

Supporting

Mentioning

152

Contrasting

Order By: Relevance

“…A major reason that dimension plays only a minor role when we consider embeddings into l 2 is the following theorem which shows that in l 2 the dimension can be significantly reduced without significant loss in distortion. (We note that the analogous statement for l 1 does not hold; see [BC03,LN04].) Theorem 13.3 (Johnson-Lindenstrauss [JL84]).…”

Section: Metric Embeddingmentioning

confidence: 95%

Expander graphs and their applications

Hoory

Linial

2006

Bull. Amer. Math. Soc.

1,502

1,355

View full text Add to dashboard Cite

A major consideration we had in writing this survey was to make it accessible to mathematicians as well as to computer scientists, since expander graphs, the protagonists of our story, come up in numerous and often surprising contexts in both fields.But, perhaps, we should start with a few words about graphs in general. They are, of course, one of the prime objects of study in Discrete Mathematics. However, graphs are among the most ubiquitous models of both natural and human-made structures. In the natural and social sciences they model relations among species, societies, companies, etc. In computer science, they represent networks of communication, data organization, computational devices as well as the flow of computation, and more. In mathematics, Cayley graphs are useful in Group Theory. Graphs carry a natural metric and are therefore useful in Geometry, and though they are "just" one-dimensional complexes, they are useful in certain parts of Topology, e.g. Knot Theory. In statistical physics, graphs can represent local connections between interacting parts of a system, as well as the dynamics of a physical process on such systems.The study of these models calls, then, for the comprehension of the significant structural properties of the relevant graphs. But are there nontrivial structural properties which are universally important? Expansion of a graph requires that it is simultaneously sparse and highly connected. Expander graphs were first defined by Bassalygo and Pinsker, and their existence first proved by Pinsker in the early '70s. The property of being an expander seems significant in many of these mathematical, computational and physical contexts. It is not surprising that expanders are useful in the design and analysis of communication networks. What is less obvious is that expanders have surprising utility in other computational settings such as in the theory of error correcting codes and the theory of pseudorandomness. In mathematics, we will encounter e.g. their role in the study of metric embeddings, and in particular in work around the Baum-Connes Conjecture. Expansion is closely related to the convergence rates of Markov Chains, and so they play a key role in the study of Monte-Carlo algorithms in statistical mechanics and in a host of practical computational applications. The list of such interesting and fruitful connections goes on and on with so many applications we will not even be able to mention. This universality of expanders is becoming more evident as more connections are discovered. It transpires that expansion is a fundamental mathematical concept, well deserving to be thoroughly investigated on its own.In hindsight, one reason that expanders are so ubiquitous is that their very definition can be given in at least three languages: combinatorial/geometric, probabilistic and algebraic. Combinatorially, expanders are graphs which are highly connected; to disconnect a large part of the graph, one has to sever many edges. Equivalently, using the geometric notion of isoperimetry, every s...

show abstract

Section: Metric Embeddingmentioning

confidence: 95%

Expander graphs and their applications

Hoory

Linial

2006

Bull. Amer. Math. Soc.

1,502

1,355

View full text Add to dashboard Cite

show abstract

“…It has been shown that a stronger guarantee, such as preserving distances to within a 1 ± ǫ factor with high probability, as given by Johnson-Lindenstrauss, does not exist for ℓ 1 distance. 38 To the best of our knowledge, the NN matching performance for quantized versions of these embeddings has not yet been reported.…”

Section: Extension To Nn Search Based On Non-euclidean Distance Measuresmentioning

confidence: 99%

Quantized embeddings: an efficient and universal nearest neighbor method for cloud-based image retrieval

2013

View full text Add to dashboard Cite

We propose a rate-efficient, feature-agnostic approach for encoding image features for cloudbased nearest neighbor search. We extract quantized random projections of the image features under consideration, transmit these to the cloud server, and perform matching in the space of the quantized projections. The advantage of this approach is that, once the underlying feature extraction algorithm is chosen for maximum discriminability and retrieval performance (e.g., SIFT, or eigen-features), the random projections guarantee a rate-efficient representation and fast server-based matching with negligible loss in accuracy. Using the Johnson-Lindenstrauss Lemma, we show that pair-wise distances between the underlying feature vectors are preserved in the corresponding quantized embeddings. We report experimental results of image retrieval on two image databases with different feature spaces; one using SIFT features and one using face features extracted using a variant of the Viola-Jones face recognition algorithm. For both feature spaces, quantized embeddings enable accurate image retrieval SPIE Conference on Applications of Digital Image Processing XXXVIThis work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. ABSTRACTWe propose a rate-efficient, feature-agnostic approach for encoding image features for cloud-based nearest neighbor search. We extract quantized random projections of the image features under consideration, transmit these to the cloud server, and perform matching in the space of the quantized projections. The advantage of this approach is that, once the underlying feature extraction algorithm is chosen for maximum discriminability and retrieval performance (e.g., SIFT, or eigen-features), the random projections guarantee a rate-efficient representation and fast server-based matching with negligible loss in accuracy. Using the Johnson-Lindenstrauss Lemma, we show that pair-wise distances between the underlying feature vectors are preserved in the corresponding quantized embeddings. We report experimental results of image retrieval on two image databases with different feature spaces; one using SIFT features and one using face features extracted using a variant of the Viola-Jones face recognition algorithm. For both feature spaces, quantized embeddings enable accurate image retrieval combined with improved bit-rate efficiency and speed of matching, when compared with the underlying featur...

show abstract

“…How about l1 as target metric? Then the situation is even worse, as Brinkman and Charikar [11] proves that for every n, the embedding of an n-point l1 metric into 1 Regarding l  as the target metric, Matousek [12] showed that any n-point metric can be embedded into d l  With distortion c and of dimension Furthermore, an almost matching lower bound can be proved, so the target dimension cannot remain bounded for a general…”

Section: mentioning

confidence: 99%

Optimizing Massive Distance Computations in Pattern Recognition

Faragó¹

2014

IJMLC

View full text Add to dashboard Cite

Abstract-It is a common task in pattern recognition to evaluate the similarity of large data objects. These are often represented by high dimensional vectors. A frequently used mathematical model for evaluating their similarity is to view them as points (vectors) in a high dimensional space, and compute their distances from each other. The "distance," however, can be defined in a very complicated way, it may be much more complex than the well known Euclidean distance. Therefore, the algorithmic bottleneck often becomes the number of distance computations that need to be carried out. We consider the case when we have to compute all the distances between n objects, where n is large. Without any shortcuts it takes n(n − 1)/2 = O(n 2 ) distance computations. In those applications where the distances are complicated, being defined by sophisticated algorithms (such as in speech and image recognition), a quadratically growing number of distance computations becomes a severe bottleneck. We prove the following general result that can help eliminating the bottleneck: for a large and general class of distances it is possible to obtain a very close approximation of each of the O(n 2 ) pairwise distances of n objects by doing only a linear number distance computations, which is optimal with respect to the order of magnitude. Moreover, the approximation factor can be made arbitrarily close to 1, making the approximation error negligible. The needed side computations to achieve this reduction can also be done in polynomial time.

show abstract

On the impossibility of dimension reduction in l ₁

Cited by 113 publications

References 27 publications

Expander graphs and their applications

Expander graphs and their applications

Quantized embeddings: an efficient and universal nearest neighbor method for cloud-based image retrieval

Optimizing Massive Distance Computations in Pattern Recognition

Contact Info

Product

Resources

About

On the impossibility of dimension reduction in l 1

Cited by 113 publications

References 27 publications

Expander graphs and their applications

Expander graphs and their applications

Quantized embeddings: an efficient and universal nearest neighbor method for cloud-based image retrieval

Optimizing Massive Distance Computations in Pattern Recognition

Contact Info

Product

Resources

About

On the impossibility of dimension reduction in l ₁