In this paper we discuss a relation between Learning Theory and Regularization of linear ill-posed inverse problems. It is well known that Tikhonov regularization can be profitably used in the context of supervised learning, where it usually goes under the name of regularized least-squares algorithm. Moreover, the gradient descent algorithm was studied recently, which is an analog of Landweber regularization scheme. In this paper we show that a notion of regularization defined according to what is usually done for ill-posed inverse problems allows to derive learning algorithms which are consistent and provide a fast convergence rate. It turns out that for priors expressed in term of variable Hilbert scales in reproducing kernel Hilbert spaces our results for Tikhonov regularization match those in Smale and Zhou [Learning theory estimates via integral operators and their approximations, submitted for publication, retrievable at http://www.tti-c.org/smale.html , 2005] and improve the results for Landweber iterations obtained in Yao et al. [On early stopping in gradient descent learning, Constructive Approximation (2005), submitted for publication]. The remarkable fact is that our analysis shows that the same properties are shared by a large class of learning algorithms which are essentially all the linear regularization schemes. The concept of operator monotone functions turns out to be an important tool for the analysis.

We prove the Li-Yau gradient estimate for the heat kernel on graphs. The only assumption is a variant of the curvature-dimension inequality, which is purely local, and can be considered as a new notion of curvature for graphs. We compute this curvature for lattices and trees and conclude that it behaves more naturally than the already existing notions of curvature. Moreover, we show that if a graph has nonnegative curvature then it has polynomial volume growth.We also derive Harnack inequalities and heat kernel bounds from the gradient estimate, and show how it can be used to strengthen the classical Buser inequality relating the spectral gap and the Cheeger constant of a graph.

Abstract. We prove the following estimate for the spectrum of the normalized Laplace operator Δ on a finite graph G,Here k[t] is a lower bound for the Ollivier-Ricci curvature on the neighborhood graph G [t], which was introduced by Bauer-Jost. In particular, when t = 1 this is Ollivier'sFor sufficiently large t, we show that, unless G is bipartite, our estimates for λ 1 and λ N −1 are always nontrivial and improve Ollivier's estimates for all graphs with k ≤ 0. By definition neighborhood graphs are weighted graphs which may have loops. To understand the Ollivier-Ricci curvature on neighborhood graphs, we generalize a sharp estimate of the Ricci curvature given by Jost-Liu to weighted graphs with loops and relate it to the relative local frequency of triangles and loops.

In the literature on regularization, many different parameter choice methods have been proposed in both deterministic and stochastic settings. However, based on the available information, it is not always easy to know how well a particular method will perform in a given situation and how it compares to other methods. This paper reviews most of the existing parameter choice methods, and evaluates and compares them in a large simulation study for spectral cut-off and Tikhonov regularization. The test cases cover a wide range of linear inverse problems with both white and colored stochastic noise. The results show some marked differences between the methods, in particular, in their stability with respect to the noise and its type. We conclude with a table of properties of the methods and a summary of the simulation results, from which we identify the best methods.

No abstract

We investigate an a posteriori stopping rule of Lepskij-type for a class of regularized Newton methods and show that it leads to order optimal convergence rates for Hölder and logarithmic source conditions without a priori knowledge of the smoothness of the solution. Numerical experiments show that this stopping rule yields results at least as good as, and in some situations significantly better than, Morozov's discrepancy principle.

No abstract

No abstract