Stochastic mirror descent method for linear ill-posed problems in Banach spaces

Abstract

Consider linear ill-posed problems governed by the system $A_i x = y_i$ for $i =1, \cdots, p$, where each $A_i$ is a bounded linear operator from a Banach space $X$ to a Hilbert space $Y_i$. In case $p$ is huge, solving the problem by an iterative regularization 
method using the whole information at each iteration step can be very expensive, due to the huge amount of memory and excessive computational load per iteration. To solve such large-scale ill-posed systems efficiently, we develop in this paper a stochastic mirror descent method which uses only a small portion of randomly selected equations at each iteration step and incorporates convex regularization terms into the algorithm design. Therefore, our method scales very well with the problem size and has the capability of capturing features of sought solutions. The convergence property of the method depends crucially on the choice of step-sizes. We consider various rules for choosing step-sizes and obtain convergence results under {\it a priori} early stopping rules. In particular, by incorporating the spirit of the discrepancy principle we propose a choice rule of step-sizes which can efficiently suppress the 
oscillations in iterates and reduce the effect of semi-convergence. Furthermore, we establish an order optimal convergence rate result when the sought solution satisfies a benchmark source condition. Various numerical simulations are reported to test the performance of the method.

Stochastic linear regularization methods: random discrepancy principle and applications

“…From (13), the second and third terms on the right side of ( 17) are bounded by 3s 2 γ 2 δ 2 + 3sη 2 δ 2 which tends to zero as δ → 0. This proves (16) and thus (11).…”

“…An extended error estimation of SGD is given in [18]. SGDs for nonlinear ill-posed operator equations are investigated in [15,16]. A second motivation for stochastic regularization comes from the uncertainty quantification of the error estimation of inverse problems.…”

Dual gradient method for ill-posed problems using multiple repeated measurement data

“…is given by x := e ℓ / ´D e ℓ , see [26]. Therefore the dual gradient method (1.5) using multiple repeated measurement data takes the form…”