We study minimization of a structured objective function, being the sum of a smooth function and a composition of a weakly convex function with a linear operator. Applications include image reconstruction problems with regularizers that introduce less bias than the standard convex regularizers. We develop a variable smoothing algorithm, based on the Moreau envelope with a decreasing sequence of smoothing parameters, and prove a complexity of $${\mathcal {O}}(\epsilon ^{-3})$$ O ( ϵ - 3 ) to achieve an $$\epsilon $$ ϵ -approximate solution. This bound interpolates between the $${\mathcal {O}}(\epsilon ^{-2})$$ O ( ϵ - 2 ) bound for the smooth case and the $${\mathcal {O}}(\epsilon ^{-4})$$ O ( ϵ - 4 ) bound for the subgradient method. Our complexity bound is in line with other works that deal with structured nonsmoothness of weakly convex functions.

Motivated by the training of Generative Adversarial Networks (GANs), we study methods for solving minimax problems with additional nonsmooth regularizers. We do so by employing monotone operator theory, in particular the Forward-Backward-Forward (FBF) method, which avoids the known issue of limit cycling by correcting each update by a second gradient evaluation. Furthermore, we propose a seemingly new scheme which recycles old gradients to mitigate the additional computational cost. In doing so we rediscover a known method, related to Optimistic Gradient Descent Ascent (OGDA). For both schemes we prove novel convergence rates for convex-concave minimax problems via a unifying approach. The derived error bounds are in terms of the gap function for the ergodic iterates. For the deterministic and the stochastic problem we show a convergence rate of O( 1 /k) and O( 1 / √ k), respectively. We complement our theoretical results with empirical improvements in the training of Wasserstein GANs on the CIFAR10 dataset.Preprint. Under review.

We investigate a structured class of nonconvex-nonconcave min-max problems exhibiting so-called weak Minty solutions, a notion which was only recently introduced, but is able to simultaneously capture different generalizations of monotonicity. We prove novel convergence results for a generalized version of the optimistic gradient method (OGDA) in this setting matching the ones recently shown for the extragradient method (EG). In addition we propose an adaptive stepsize version of EG, which does not require knowledge of the problem parameters.

No abstract

Schröder iteration functions, a generalization of the Newton-Raphson method to determine roots of equations, are generally rational functions which possess some critical points free to converge to attracting cycles. These free critical points, however, satisfy some higher-degree polynomial equations. We present a new algorithmic construction to compute in general all of the Schröder functions' terms as well as to maximize the computational efficiency of these functions associated with a one-parameter family of cubic polynomials. Finally, we examine the Julia sets of the Schröder functions constructed to converge to the nth roots of unity, these roots' basins of attraction, and the orbits of all free critical points of these functions for order higher than four, as applied to the one-parameter family of cubic polynomials mentioned above. Introduction.A number of mathematical applications lead to the problem of finding the roots of some equation using iterative methods. Unfortunately, the convergence of an iterative method is not assured independently of the starting value. There exist certain starting values that are not suitable in yielding the desired result. Looking at the set of all the starting values from a geometrical point of view will definitely help us to choose these initial approximations to a root and compare the convergence properties of various iterative methods.Ernst Schröder [12] in 1870 (see also [7]) described a method of finding a rational iterating function of any desired "order of convergence" to determine roots of equations. For polynomial equations this involves the iteration of rational functions over the Riemann sphere which is described by the classical theory of Julia [9] and Fatou [6] and its subsequent developments, also of paramount importance in the context of numerical analysis. In what follows we abbreviate as f k the k-fold composition f • f • · · · • f , by region we mean a connected open set on the extended complex plane C = C ∪ {∞}, and if we have a rational function of the form R = P/Q, where P and Q are complex polynomials with no common factors, the degree of R is defined by deg(R) = max{deg(P ), deg(Q)}.It appears that, for cases of practical interest, convergence of the sequence of iterates

We aim to solve a structured convex optimization problem, where a nonsmooth function is composed with a linear operator. When opting for full splitting schemes, usually, primal–dual type methods are employed as they are effective and also well studied. However, under the additional assumption of Lipschitz continuity of the nonsmooth function which is composed with the linear operator we can derive novel algorithms through regularization via the Moreau envelope. Furthermore, we tackle large scale problems by means of stochastic oracle calls, very similar to stochastic gradient techniques. Applications to total variational denoising and deblurring, and matrix factorization are provided.

No abstract

We investigate the convergence properties of incremental mirror descent type subgradient algorithms for minimizing the sum of convex functions. In each step, we only evaluate the subgradient of a single component function and mirror it back to the feasible domain, which makes iterations very cheap to compute. The analysis is made for a randomized selection of the component functions, which yields the deterministic algorithm as a special case. Under supplementary differentiability assumptions on the function which induces the mirror map, we are also able to deal with the presence of another term in the objective function, which is evaluated via a proximal type step. In both cases, we derive convergence rates of in expectation for the k th best objective function value and illustrate our theoretical findings by numerical experiments in positron emission tomography and machine learning.