Dimension Reduction by Random Hyperplane Tessellations

Plan, Yaniv; Vershynin, Roman

doi:10.1007/s00454-013-9561-6

Cited by 99 publications

(143 citation statements)

References 24 publications

Supporting

Mentioning

141

Contrasting

Order By: Relevance

“…We begin with the following simple comparison, which follows from the Cauchy-Schwartz inequality for all v ∈ R n : (Indeed, u = tv/ v 2 lies in K since t/ v ≤ 1 and K is star shaped.) Next, we will control Au 1 with an application of the following uniform deviation inequality, which we proved in [35].…”

Section: Proof Of Main Resultsmentioning

confidence: 99%

The Generalized Lasso With Non-Linear Observations

Plan

Vershynin

2016

IEEE Trans. Inform. Theory

Self Cite

154

196

View full text Add to dashboard Cite

Abstract. We study the problem of signal estimation from non-linear observations when the signal belongs to a low-dimensional set buried in a high-dimensional space. A rough heuristic often used in practice postulates that non-linear observations may be treated as noisy linear observations, and thus the signal may be estimated using the generalized Lasso. This is appealing because of the abundance of efficient, specialized solvers for this program. Just as noise may be diminished by projecting onto the lower dimensional space, the error from modeling non-linear observations with linear observations will be greatly reduced when using the signal structure in the reconstruction. We allow general signal structure, only assuming that the signal belongs to some set K ⊂ R n . We consider the single-index model of non-linearity. Our theory allows the non-linearity to be discontinuous, not one-to-one and even unknown. We assume a random Gaussian model for the measurement matrix, but allow the rows to have an unknown covariance matrix. As special cases of our results, we recover near-optimal theory for noisy linear observations, and also give the first theoretical accuracy guarantee for 1-bit compressed sensing with unknown covariance matrix of the measurement vectors.

show abstract

Section: Proof Of Main Resultsmentioning

confidence: 99%

The Generalized Lasso With Non-Linear Observations

Plan

Vershynin

2016

IEEE Trans. Inform. Theory

Self Cite

154

196

View full text Add to dashboard Cite

show abstract

“…On the other hand, is a function of K, the projection's dimensionality, and scales approximately as 1/ √ K. In the extreme case of 1-bit scalar quantization the embedding does not preserve signal amplitudes and, therefore, their 2 distances. Still, it does preserve their angle, i.e., their correlation coefficient [17], [18].…”

Section: Background a Randomized Embeddingsmentioning

confidence: 99%

Efficient Coding of Signal Distances Using Universal Quantized Embeddings

Boufounos

Rane

2013

2013 Data Compression Conference

View full text Add to dashboard Cite

Traditional rate-distortion theory is focused on how to best encode a signal using as few bits as possible and incurring as low a distortion as possible. However, very often, the goal of transmission is to extract specific information from the signal at the receiving end, and the distortion should be measured on that extracted information. In this paper we examine the problem of encoding signals such that sufficient information is preserved about their pairwise distances. For that goal, we consider randomized embeddings as an encoding mechanism and provide a framework to analyze their performance. We also propose the recently developed universal quantized embeddings as a solution to that problem and experimentally demonstrate that, in image retrieval experiments, universal embedding can achieve up to 25% rate reduction over the state of the art.I. INTRODUCTION Source coding theory and practice has primarily focused on how to best encode a signal for transmission using the fewest possible bits while incurring the smallest possible distortion. For example, in image or video compression, the encoder aims to reduce the bit-rate for a given visual reconstruction quality. This goal is dictated by the end user of the signal: an image or a video will be viewed by a human being. Quite often, however, the end user of a signal is not a human being observing the distorted signal per se, but a server extracting information. In this case, the goal is different: encoding must happen in a way that does not destroy the information that the server wants to extract, even if the signal itself cannot be completely recovered. In particular, we examine applications in which the server is interested in extracting only the information about the distance of a signal from its nearest neighbors. This paper examines how to efficiently encode signals for transmission such that the receiver can approximately determine the distance between signals up to a specified radius. Our encoding exploits the recently developed theory for efficient universal quantization and universal quantized embeddings [1]. We demonstrate that, using universal quantized embeddings, we are able to improve compression performance up to 25% over previous embedding-based approaches [2], [3], including our own earlier work [4]. The main advantage of universal embeddings is that they preserve distance information only up to a certain radius, as required to determine the near neighbors, and not any farther. Thus, rate is not wasted in coding distances larger than necessary.Our main-but not the only-motivating example is image retrieval, with emphasis on augmented reality (AR) applications. As we discuss in [4], AR and more general image retrieval applications can benefit significantly by efficient coding of distances to a signal's nearest neighbors. In typical cloud-based image retrieval applications, a client transmits

show abstract

“…Tighter bounds can instead be developed if we are interested in preservation of the angles between two signals-i.e., their correlation or inner productinstead of the distance between them. [30][31][32] …”

Section: Theorem 1 (Johnson-lindenstrauss Lemmamentioning

confidence: 99%

Quantized embeddings: an efficient and universal nearest neighbor method for cloud-based image retrieval

2013

View full text Add to dashboard Cite

We propose a rate-efficient, feature-agnostic approach for encoding image features for cloudbased nearest neighbor search. We extract quantized random projections of the image features under consideration, transmit these to the cloud server, and perform matching in the space of the quantized projections. The advantage of this approach is that, once the underlying feature extraction algorithm is chosen for maximum discriminability and retrieval performance (e.g., SIFT, or eigen-features), the random projections guarantee a rate-efficient representation and fast server-based matching with negligible loss in accuracy. Using the Johnson-Lindenstrauss Lemma, we show that pair-wise distances between the underlying feature vectors are preserved in the corresponding quantized embeddings. We report experimental results of image retrieval on two image databases with different feature spaces; one using SIFT features and one using face features extracted using a variant of the Viola-Jones face recognition algorithm. For both feature spaces, quantized embeddings enable accurate image retrieval SPIE Conference on Applications of Digital Image Processing XXXVIThis work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. ABSTRACTWe propose a rate-efficient, feature-agnostic approach for encoding image features for cloud-based nearest neighbor search. We extract quantized random projections of the image features under consideration, transmit these to the cloud server, and perform matching in the space of the quantized projections. The advantage of this approach is that, once the underlying feature extraction algorithm is chosen for maximum discriminability and retrieval performance (e.g., SIFT, or eigen-features), the random projections guarantee a rate-efficient representation and fast server-based matching with negligible loss in accuracy. Using the Johnson-Lindenstrauss Lemma, we show that pair-wise distances between the underlying feature vectors are preserved in the corresponding quantized embeddings. We report experimental results of image retrieval on two image databases with different feature spaces; one using SIFT features and one using face features extracted using a variant of the Viola-Jones face recognition algorithm. For both feature spaces, quantized embeddings enable accurate image retrieval combined with improved bit-rate efficiency and speed of matching, when compared with the underlying featur...

show abstract

Dimension Reduction by Random Hyperplane Tessellations

Cited by 99 publications

References 24 publications

The Generalized Lasso With Non-Linear Observations

The Generalized Lasso With Non-Linear Observations

Efficient Coding of Signal Distances Using Universal Quantized Embeddings

Quantized embeddings: an efficient and universal nearest neighbor method for cloud-based image retrieval

Contact Info

Product

Resources

About