Stochastic linear regularization methods: random discrepancy principle and applications

Abstract

The a posteriori stopping rule plays a significant role in the design of efficient stochastic algorithms for various tasks in computational mathematics, such as inverse problems, optimization, and machine learning. Through the lens of classical regularization theory, this paper describes a novel analysis of Morozov’s discrepancy principle for the stochastic generalized Landweber iteration and its continuous analog of generalized stochastic asymptotical regularization. Unlike existing results relating to convergence in probability, we prove the strong convergence of the regularization error using tools from stochastic analysis, namely the theory of martingales. Numerical experiments are conducted to verify the convergence of the discrepancy principle and demonstrate two new capabilities of stochastic generalized Landweber iteration, which should also be valid for other stochastic/statistical approaches: improved accuracy by selecting the optimal path and the identification of multi-solutions by clustering samples of obtained approximate solutions.

A posterior contraction for Bayesian inverse problems in Banach spaces

“…It is well known that when employing a nonparametric Bayesian approach with standard Gaussian priors, the posterior-based reconstruction corresponds to a Tikhonov regularizer with a reproducing kernel Hilbert space (RKHS) norm penalty [22, section 7.1]. Recently, by combining conventional regularization and statistical formulation, some stochastic regularization methods with novel features have been discussed in [45,46]. Therefore, although the posterior x, obtained by setting prior variance based on the knowledge of the noise level, does not belong to the conventional Bayesian paradigm, it can serve as a good estimation of x † as interpreted above.…”