Fast Binary Embeddings and Quantized Compressed Sensing with Structured Matrices

Abstract

This paper deals with two related problems, namely distance‐preserving binary embeddings and quantization for compressed sensing. First, we propose fast methods to replace points from a subset Χ ⊂ ℝn, associated with the euclidean metric, with points in the cube {±1}m, and we associate the cube with a pseudometric that approximates euclidean distance among points in Χ. Our methods rely on quantizing fast Johnson‐Lindenstrauss embeddings based on bounded orthonormal systems and partial circulant ensembles, both of which admit fast transforms. Our quantization methods utilize noise shaping, and include sigma‐delta schemes and distributed noise‐shaping schemes. The resulting approximation errors decay polynomially and exponentially fast in m, depending on the embedding method. This dramatically outperforms the current decay rates associated with binary embeddings and Hamming distances. Additionally, it is the first such binary embedding result that applies to fast Johnson‐Lindenstrauss maps while preserving ℓ2 norms. Second, we again consider noise‐shaping schemes, albeit this time to quantize compressed sensing measurements arising from bounded orthonormal ensembles and partial circulant matrices. We show that these methods yield a reconstruction error that again decays with the number of measurements (and bits), when using convex optimization for reconstruction. Specifically, for sigma‐delta schemes, the error decays polynomially in the number of measurements, and it decays exponentially for distributed noise‐shaping schemes based on beta encoding. These results are near optimal and the first of their kind dealing with bounded orthonormal systems. © 2019 Wiley Periodicals, Inc.

New Algorithms and Improved Guarantees for One-Bit Compressed Sensing on Manifolds

“…This simple algorithm first finds a GMRA center that quantizes to a bit sequence close to the quantized measurements, where "closeness" is determined using a pseudometric that respects the quantization; it then optimizes over all points in the associated approximate tangent space to enforce, as much as possible, the consistency of the quantization. Using the results of [8] we prove that the quantization error associated with our proposed reconstruction algorithm decays polynomially or exponentially as a function of the number of measurements, depending on the quantization scheme. This greatly improves on the sub-linear error decay associated with scalar quantization in [9].…”

“…Thus, it is necessary to consider the effect of quantization in the design of the recovery algorithms. Indeed, sparse vector recovery and low-rank matrix recovery have been studied in the presence of various quantization schemes [7], [8], [11], [12], [13]. We look to extend these results to account for those structured signals that lie on a compact, lowdimensional submanifold of R N for which we have a Geometric Multi-Resolution Analysis (GMRA) [1].…”

“…At each level the GMRA is a collection of approximate tangent spaces about certain known "centers", and the quality of the approximation improves with every level. Unlike in [9], the quantization schemes that we use are Σ∆ or distributed noise shaping methods (see, e.g., [7], [8]) and the compressed sensing measurements that our results apply to include those drawn from Gaussian ensembles, partial circulant ensembles (PCE) or bounded orthonormal ensembles (BOE) (see [8] for precise definitions). Our proposed reconstruction method is summarized in Algorithm 1.…”

Adapted Decimation on Finite Frames for Arbitrary Orders of Sigma-Delta Quantization

“…Iwen and Saab [23] used probabilistic arguments and the property of efficient storage to construct random quantization schemes with exponential error decay rate with respect to the bit usage. In [21], similar ideas are used on Σ∆. Moreover, the connection between decimation and distributed noise shaping can be seen in it.…”

“…Moreover, the connection between decimation and distributed noise shaping can be seen in it. [23,21] both use probabilistic arguments that only ensure success with some probability instead of deterministic guarantee. For the explicit and deterministic adaptation to finite dimensional signals, the author proved in [24] that there exists a similar operator called the alternative decimation operator that behaves similarly to the decimation for bandlimited signals.…”