Structured Random Sketching for PDE Inverse Problems

Chen, Ke; Li, Qin; Newton, Kit; Wright, Stephen J.

doi:10.1137/20m1310497

Cited by 5 publications

(8 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sketching designs of row-wise tensor structure are previously studied in [40,10,37] with applications for data memory reductions, tensor decompositions and linear inverse problems. Through the lens of unconstrained linear regressions, by applying a standard covering net technique, their results are suboptimal.…”

Section: Related Workmentioning

confidence: 99%

“…where d + 1 is the order of the original tensor from which A is unfolded. Due to the high computation cost of solving (10), sketching is used as a tool of solving CP tensor decomposition, see [6].…”

Section: Design Of the Sketching Matrixmentioning

confidence: 99%

“…This design is natural to process the tensor data in the program. Previous works that share similar set-ups are [40,10,37]. We include the details of comparisons in later discussions.…”

Section: Introductionmentioning

confidence: 99%

“…This question has been well investigated in a line of work [17,38,29,12,3,35,36] based on common dimensionality reduction methods like CountSketch [9], sparse 0-1 matrices [14], Gaussian and sub-Gaussian matrices [39,28], and FFT-based fast constructions [1,44]. In many applications, the coefficient data A, b admit multi-linear (tensor) structures, for example in: spatio-temporal data analysis [4,19], higher-order tensor decompositions [15,24], approximating polynomial kernels [22,34], linearized PDE inverse problems [10,32] and so on. In the sketching setting, we can utilize the tensor structure in the original objective function to speed up forming the sketched problem by making the sketching matrix have a corresponding tensor structure.…”

Section: Introductionmentioning

confidence: 99%

“…[25] analyzes sparse sketching techniques in low-rank tensor regressions for tensor CP and Tucker decompositions. For computational advantage from the tensor structure in the sketching matrices, please see the works [10,34,20] for details.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Tensor-structured sketching for constrained least squares

Chen¹,

Jin²

2020

Preprint

Self Cite

View full text Add to dashboard Cite

In this work, we study a tensor-structured random sketching matrix to project a largescale convex optimization problem to a much lower-dimensional counterpart, which leads to huge memory and computation savings. We show that while maintaining the prediction error between a random estimator and the true solution with high probability, the dimension of the projected problem obtains optimal dependence in terms of the geometry of the constraint set. Moreover, the tensor structure and sparsity pattern of the structured random matrix yields extra computational advantage. Our analysis is based on probability chaining theory, which allows us to obtain an almost sharp estimate for the sketching dimension of convex optimization problems. Consequences of our main result are demonstrated in a few concrete examples, including unconstrained linear regressions and sparse recovery problems.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Design Of the Sketching Matrixmentioning

confidence: 99%

“…This design is natural to process the tensor data in the program. Previous works that share similar set-ups are [40,10,37]. We include the details of comparisons in later discussions.…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Tensor-structured sketching for constrained least squares

Chen¹,

Jin²

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

On unifying randomized methods for inverse problems

et al. 2023

View full text Add to dashboard Cite

This work unifies the analysis of various randomized methods for solving linear and nonlinear in-verse problems by framing the problem in a stochastic optimization setting. By doing so, we showthat many randomized methods are variants of a sample average approximation. More importantly,we are able to prove a single theoretical result that guarantees the asymptotic convergence for avariety of randomized methods. Additionally, viewing randomized methods as a sample average ap-proximation enables us to prove, for the first time, a single non-asymptotic error result that holds forrandomized methods under consideration. Another important consequence of our unified frameworkis that it allows us to discover new randomization methods. We present various numerical resultsfor linear, nonlinear, algebraic, and PDE-constrained inverse problems that verify the theoreticalconvergence results and provide a discussion on the apparently different convergence rates and thebehavior for various randomized methods.

show abstract

Sketched and truncated polynomial Krylov subspace methods: Matrix Sylvester equations

Palitta,

Schweitzer,

Simoncini

2024

Math. Comp.

View full text Add to dashboard Cite

Thanks to its great potential in reducing both computational cost and memory requirements, combining sketching and Krylov subspace techniques has attracted a lot of attention in the recent literature on projection methods for linear systems, matrix function approximations, and eigenvalue problems. Applying this appealing strategy in the context of linear matrix equations turns out to be far more involved than a straightforward generalization. These difficulties include analyzing well-posedness of the projected problem and deriving possible error estimates depending on the sketching properties. Further computational complications include the lack of a natural residual norm estimate and of an explicit basis for the generated subspace. In this paper we propose a new sketched-and-truncated polynomial Krylov subspace method for Sylvester equations that aims to address all these issues. The potential of our novel approach, in terms of both computational time and storage demand, is illustrated with numerical experiments. Comparisons with a state-of-the-art projection scheme based on rational Krylov subspaces are also included.

show abstract

Structured Random Sketching for PDE Inverse Problems

Cited by 5 publications

References 19 publications

Tensor-structured sketching for constrained least squares

Tensor-structured sketching for constrained least squares

On unifying randomized methods for inverse problems

Sketched and truncated polynomial Krylov subspace methods: Matrix Sylvester equations

Contact Info

Product

Resources

About