Compression-Based Compressed Sensing

Abstract

Modern compression algorithms exploit complex structures that are present in signals to describe them very efficiently. On the other hand, the field of compressed sensing is built upon the observation that "structured" signals can be recovered from their under-determined set of linear projections. Currently, there is a large gap between the complexity of the structures studied in the area of compressed sensing and those employed by the state-of-the-art compression codes. Recent results in the literature on deterministic signals aim at bridging this gap through devising compressed sensing decoders that employ compression codes. This paper focuses on structured stochastic processes and studies the application of rate-distortion codes to compressed sensing of such signals. The performance of the formerly-proposed compressible signal pursuit (CSP) algorithm is studied in this stochastic setting. It is proved that in the very low distortion regime, as the blocklength grows to infinity, the CSP algorithm reliably and robustly recovers n instances of a stationary process from random linear projections as long as their count is slightly more than n times the rate-distortion dimension (RDD) of the source. It is also shown that under some regularity conditions, the RDD of a stationary process is equal to its information dimension (ID). This connection establishes the optimality of the CSP algorithm at least for memoryless stationary sources, for which the fundamental limits are known. Finally, it is shown that the CSP algorithm combined by a family of universal variable-length fixed-distortion compression codes yields a family of universal compressed sensing recovery algorithms.

On the Information Dimension of Stochastic Processes

“…The essential difference between the definitions of d({X t }) and d ′ ({X t }) is the order in which the limits over the quantization bin size 1/m and the block size k are taken. Rezagah et al [8] showed that these limits can be exchanged if the process satisfies…”

“…8]. Rezagah et al [8] showed that d ′ ({X t }) coincides, under certain conditions, with the rate-distortion dimension dim R ({X t }), thus generalizing the result by Kawabata and Dembo [2] to stochastic processes.…”

“…The connection between the block-average information dimension and the rate-distortion dimension of a stochastic process was studied in [8]. The equivalence between the rate-distortion dimension and the information dimension…”

“…Rezagah et al [8] demonstrated that the order of the limits on the RHS of (115a) and (115b) can be exchanged.…”

On the information dimension rate of stochastic processes

“…8]. Rezagah et al showed in [5], [6] that d 0 (X) coincides, under certain conditions, with the rate-distortion dimension dim R ({X t }), defined as twice the rate-distortion function of the stochastic process {X t , t 2 Z} divided by log D in the limit as D # 0. This generalizes to stochastic processes the result by Kawabata and Dembo that the rate-distortion dimension of a random variable (RV) equals its information dimension [7].…”