From Compressed Sensing to Compressed Bit-Streams: Practical Encoders, Tractable Decoders

Abstract

Compressed sensing is now established as an effective method for dimension reduction when the underlying signals are sparse or compressible with respect to some suitable basis or frame. One important, yet under-addressed problem regarding the compressive acquisition of analog signals is how to perform quantization. This is directly related to the important issues of how "compressed" compressed sensing is (in terms of the total number of bits one ends up using after acquiring the signal) and ultimately whether compressed sensing can be used to obtain compressed representations of suitable signals. Building on our recent work, we propose a concrete and practicable method for performing "analog-to-information conversion" . Following a compressive signal acquisition stage, the proposed method consists of a quantization stage, based on Σ∆ (sigma-delta) quantization, and a subsequent encoding (compression) stage that fits within the framework of compressed sensing seamlessly. We prove that, using this method, we can convert analog compressive samples to compressed digital bitstreams and decode using tractable algorithms based on convex optimization. We prove that the proposed AIC provides a nearly optimal encoding of sparse and compressible signals. Finally, we present numerical experiments illustrating the effectiveness of the proposed analog-to-information converter. Encoding (Compression):The quantized measurements require log 2 |C 0 | bits to be represented. Often, we can reduce this bit budget by incorporating an encoding stage. We denote by E : C 0 → C, the encoding map, where C is a finite set called the codebook, usually satisfying log 2 |C| ≪ log 2 |C 0 |. The goal of encoding is to reduce the number of bits while still permitting accurate reconstruction. We focus on simple encoding schemes implemented via, e.g., discrete Johnson-Lindenstrauss embeddings [3] (cf. [4, 5, 6, 7]).Reconstruction: The final stage of a compressive AIC is the reconstruction or decoding stage where we recover an approximation to the original signal x. To that end we use a map ∆ : C → R N .

Fast Binary Embeddings and Quantized Compressed Sensing with Structured Matrices

“…The proof of the above is simply by induction and can be found, for example, in [19]. Our algorithm are inspired by the approach to reconstruction from noise-shaping quantized measurements used in [17] and [62] (see also [20]). We first introduce a so-called condensation operator V : C m → C p , for some p ∈ [m], defined by…”

Quantization for low-rank matrix recovery

“…It should further be noted that the numerical distortions decay much faster than the upper bound of (28). We suspect this to be due to the sub-optimal r-dependent constants in the exponent of (28), which are likely an artifact of the proof technique in [35]. Indeed, there is evidence in [35] that the correct exponent is −r/2 + 1/4 rather than −r/2 + 3/4.…”

“…We suspect this to be due to the sub-optimal r-dependent constants in the exponent of (28), which are likely an artifact of the proof technique in [35]. Indeed, there is evidence in [35] that the correct exponent is −r/2 + 1/4 rather than −r/2 + 3/4. This is more in line with our numerical exerpiments but the proof of such is beyond the scope of this paper.…”

New Algorithms and Improved Guarantees for One-Bit Compressed Sensing on Manifolds

“…So, to acheive better error rates one must use more sophisticated quantization schemes. For example, in the sparse vector setting noise shaping techniques such as Σ∆ and distributed noise shaping leverage redundancy of the measurements to ensure error decay like O(m −r ) or O(β −cm ) for some parameters r ∈ N, β > 1 that depend on the quantization scheme, e.g., [4], [14]. As we will also use these schemes, we now briefly describe them.…”