A universal sampling method for reconstructing signals with simple Fourier transforms

Abstract

Reconstructing continuous signals based on a small number of discrete samples is a fundamental problem across science and engineering. In practice, we are often interested in signals with "simple" Fourier structure -e.g., those involving frequencies within a bounded range, a small number of frequencies, or a few blocks of frequencies. 1 More broadly, any prior knowledge about a signal's Fourier power spectrum can constrain its complexity. Intuitively, signals with more highly constrained Fourier structure require fewer samples to reconstruct.We formalize this intuition by showing that, roughly, a continuous signal from a given class can be approximately reconstructed using a number of samples proportional to the statistical dimension of the allowed power spectrum of that class. We prove that, in nearly all settings, this natural measure tightly characterizes the sample complexity of signal reconstruction.Surprisingly, we also show that, up to logarithmic factors, a universal non-uniform sampling strategy can achieve this optimal complexity for any class of signals. We present a simple, efficient, and general algorithm for recovering a signal from the samples taken. For bandlimited and sparse signals, our method matches the state-of-the-art. At the same time, it gives the first computationally and sample efficient solution to a broad range of problems, including multiband signal reconstruction and kriging and Gaussian process regression tasks in one dimension.Our work is based on a novel connection between randomized linear algebra and the problem of reconstructing signals with constrained Fourier structure. We extend tools based on statistical leverage score sampling and column-based matrix reconstruction to the approximation of continuous linear operators that arise in the signal reconstruction problem. We believe that these extensions are of independent interest and serve as a foundation for tackling a broad range of continuous time problems using randomized methods.

Spatial mapping with Gaussian processes and nonstationary Fourier features

“…In terms of KL divergence, the Nyström approximation performs best for all ranges followed by the RFF and SPDE approximations. This simple experiment demonstrates what has previously already been theoretically established; that the RFF approximation is competitive with other widely used covariance approximations Sriperumbudur and Szabo, 2015, Li and Honorio, 2017, Rahimi and Recht, 2009, Rahimi and Recht, 2008a, Rahimi and Recht, 2008b, Avron et al, 2017, Huang et al, 2014, Dai et al, 2014.…”

“…To provide some intuition to the reader we perform a simple simulation experiment in Appendix. For a more thorough treatment of the theoretical properties of RFF kernels, finite-sample performance, uniform convergence bounds, and kernel approximation quality the reader is directed here Sriperumbudur and Szabo, 2015, Li and Honorio, 2017, Rahimi and Recht, 2009, Rahimi and Recht, 2008a, Rahimi and Recht, 2008b, Avron et al, 2017.…”

Flexible rational approximation for matrix functions

“…In practice, one can use a global sampling strategy, for example, equidistant sampling, which is sufficient in many cases (as seen in the sequel). Furthermore, a universal approach for a wide class of common functions is introduced in [4], which randomly samples the segment of interest according to non-uniform ideal sampling distribution. Alternatively, when the number of points is required to be small, one may use Chebyshev points (or nodes) which are the roots of the Chebyshev polynomial of the first kind.…”

Randomized numerical linear algebra: Foundations and algorithms

“…In this setting, sampling corresponds to function evaluation, while mixing embeddings may lead to operations that are impossible to implement in the continuous space. For some examples, see (Rauhut and Ward 2012, Cohen, Davenport and Leviatan 2013, Hampton and Doostan 2015, Rauhut and Ward 2016, Cohen and Migliorati 2017, Arras, Bachmayr and Cohen 2019, Avron, Kapralov, Musco, Musco, Velingker and Zandieh 2019 and (Chen and Price 2019). 9.6.5.…”