Variable Smoothing for Weakly Convex Composite Functions

Abstract

We study minimization of a structured objective function, being the sum of a smooth function and a composition of a weakly convex function with a linear operator. Applications include image reconstruction problems with regularizers that introduce less bias than the standard convex regularizers. We develop a variable smoothing algorithm, based on the Moreau envelope with a decreasing sequence of smoothing parameters, and prove a complexity of $${\mathcal {O}}(\epsilon ^{-3})$$ O ( ϵ - 3 ) to achieve an $$\epsilon $$ ϵ -approximate solution. This bound interpolates between the $${\mathcal {O}}(\epsilon ^{-2})$$ O ( ϵ - 2 ) bound for the smooth case and the $${\mathcal {O}}(\epsilon ^{-4})$$ O ( ϵ - 4 ) bound for the subgradient method. Our complexity bound is in line with other works that deal with structured nonsmoothness of weakly convex functions.

A projected semismooth Newton method for a class of nonconvex composite programs with strong prox-regularity

“…Analogously, one can also verify the weak convexity of the SCAD regularizer and the semismoothness of its proximal operator. We refer to [7] and [48] for the details. Numerical results in [48] exhibit the efficiency of semismooth Newton methods.…”

“…It has been shown in [7] that two popular nonsmooth nonconvex regularizers, the minimax concave penalty [56] and the smoothly clipped absolute deviation [19], are weakly convex. Since any smooth manifold is proximally smooth, the manifold optimization problems [1,23,8] take the form (1.1).…”

Solving Nonconvex-Nonconcave Min-Max Problems exhibiting Weak Minty Solutions

“…Such operators arise as the (sub)-gradients of the well-studied class (see [7,20,22]) of weakly convex functions -a rather generic class of functions as it includes all functions without upward cusps.…”

“…In particular, there exists an index k 0 ∈ N such that a k /a k+1 ≤ 1/(σβ) for all k ≥ k 0 , because σ < 1/2 = β −1 . We can therefore drop the last term in (7) and sum up to obtain…”

AdAUC: End-to-end Adversarial AUC Optimization Against Long-tail Problems

“…In this sense, we then try to find a surrogate objective induce the concavity w.r.t δ i . Specifically, we adopt a concavity regularization term −γ||x i + δ i || 2 2 , and define a surrogate objective: (Liu et al, 2021;Böhm & Wright, 2021), then we can define γ > γ to obtain an objective f (w, α, x + δ) such that max α…”