Sample Complexity Lower Bounds for Compressive Sensing with Generative Models

Liu, Zhaoqiang; Scarlett, Jonathan

doi:10.1109/spcom50965.2020.9179594

Cited by 15 publications

(54 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Surprisingly, the sharp sample complexity for the squares decoder (3)-( 2) can be derived in this paper even if the measurements are highly quantized and corrupted by noise and sign flips. Very recently, under generative prior, [33] and [42] derived sample complexity results for 1-bit CS. The sample complexity obtained in [33] is O(k log L) under the assumption that the generator G is L-Lipschitz continuous and the rows of A are i.i.d.…”

Section: Related Workmentioning

confidence: 99%

“…Very recently, under generative prior, [33] and [42] derived sample complexity results for 1-bit CS. The sample complexity obtained in [33] is O(k log L) under the assumption that the generator G is L-Lipschitz continuous and the rows of A are i.i.d. sampled from N (0, I).…”

Section: Related Workmentioning

confidence: 99%

“…sampled from N (0, I). However, the estimator proposed in [33], x = G(ẑ) with ẑ ∈ {z : y = sign(AG(z))}, is quite different from our least squares decoder and the analysis technique used there are also not applicable to our decoder. [42] proposed unconstrained empirical risk minimization to recovery in 1-bit CS and derived the sample complexity to be O(kd log n) for d-layer ReLU network G via assuming the rows of A are i.i.d.…”

Section: Related Workmentioning

confidence: 99%

“…By assuming the Lipschitz constant L is larger than n (this usually holds in deep neural network generators), the estimation error O( k log L m ) and the sample complexity m ≥ O(k log L) proved in Theorem 2.2 are sharp even in the standard compressed sensing with generative prior [35]. Under generative prior, [33] proposed the estimator x = G(ẑ) with ẑ ∈ {z : y = sign(AG(z))} in the setting the rows of A are i.i.d. sampled from N (0, I).…”

Section: Analysis Of the Least Square Decodermentioning

confidence: 99%

“…sampled from N (0, I). The sample complexity obtained in [33] is also O(k log L). [42] proposed unconstrained empirical risk minimization to recover the 1-bit CS in the scenario that the rows of A are i.i.d.…”

Section: Analysis Of the Least Square Decodermentioning

confidence: 99%

See 4 more Smart Citations

Just Least Squares: Binary Compressive Sampling with Low Generative Intrinsic Dimension

Jiao

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we consider recovering n dimensional signals from m binary measurements corrupted by noises and sign flips under the assumption that the target signals have low generative intrinsic dimension, i.e., the target signals can be approximately generated via an L-Lipschitz generator G :Although the binary measurements model is highly nonlinear, we propose a least square decoder and prove that, up to a constant c, with high probability, the least square decoder achieves a sharp estimation error O( k log(Ln) m ) as long as m ≥ O(k log(Ln)). Extensive numerical simulations and comparisons with state-of-the-art methods demonstrated the least square decoder is robust to noise and sign flips, as indicated by our theory. By constructing a ReLU network with properly chosen depth and width, we verify the (approximately) deep generative prior, which is of independent interest.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Analysis Of the Least Square Decodermentioning

confidence: 99%

Section: Analysis Of the Least Square Decodermentioning

confidence: 99%

See 3 more Smart Citations

Just Least Squares: Binary Compressive Sampling with Low Generative Intrinsic Dimension

Jiao

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Compressive phase retrieval: Optimal sample complexity with deep generative priors

Hand,

Leong,

Voroninski

2023

Comm Pure Appl Math

View full text Add to dashboard Cite

Advances in compressive sensing (CS) provided reconstruction algorithms of sparse signals from linear measurements with optimal sample complexity, but natural extensions of this methodology to nonlinear inverse problems have been met with potentially fundamental sample complexity bottlenecks. In particular, tractable algorithms for compressive phase retrieval with sparsity priors have not been able to achieve optimal sample complexity. This has created an open problem in compressive phase retrieval: under generic, phaseless linear measurements, are there tractable reconstruction algorithms that succeed with optimal sample complexity? Meanwhile, progress in machine learning has led to the development of new data‐driven signal priors in the form of generative models, which can outperform sparsity priors with significantly fewer measurements. In this work, we resolve the open problem in compressive phase retrieval and demonstrate that generative priors can lead to a fundamental advance by permitting optimal sample complexity by a tractable algorithm. We additionally provide empirics showing that exploiting generative priors in phase retrieval can significantly outperform sparsity priors. These results provide support for generative priors as a new paradigm for signal recovery in a variety of contexts, both empirically and theoretically. The strengths of this paradigm are that (1) generative priors can represent some classes of natural signals more concisely than sparsity priors, (2) generative priors allow for direct optimization over the natural signal manifold, which is intractable under sparsity priors, and (3) the resulting non‐convex optimization problems with generative priors can admit benign optimization landscapes at optimal sample complexity, perhaps surprisingly, even in cases of nonlinear measurements.

show abstract

Just Least Squares: Binary Compressive Sampling with Low Generative Intrinsic Dimension

Jiao

Li²,

Liu

et al. 2023

J Sci Comput

View full text Add to dashboard Cite

Sample Complexity Lower Bounds for Compressive Sensing with Generative Models

Cited by 15 publications

References 18 publications

Just Least Squares: Binary Compressive Sampling with Low Generative Intrinsic Dimension

Just Least Squares: Binary Compressive Sampling with Low Generative Intrinsic Dimension

Compressive phase retrieval: Optimal sample complexity with deep generative priors

Just Least Squares: Binary Compressive Sampling with Low Generative Intrinsic Dimension

Contact Info

Product

Resources

About