Revisiting linearized Bregman iterations under Lipschitz-like convexity condition

Abstract

The linearized Bregman iterations (LBreI) and its variants have received considerable attention in signal/image processing and compressed sensing. Recently, LBreI has been extended to a larger class of nonconvex functions, along with several theoretical issues left for further investigation. In particular, the Lipschitz gradient continuity assumption precludes its use in many practical applications. In this study, we propose a generalized algorithmic framework to unify LBreI-type methods. Our main discovery is that the Lipschitz gradient continuity assumption can be replaced by a Lipschitz-like convexity condition in both convex and nonconvex cases. As a by-product, a class of bilevel optimization problems can be solved in the proposed framework, which extends the main result made by Cai et al. [Math. Comp. 78 (2009), pp. 2127–2136]. At last, provably convergent iterative schemes on modified linear/quadratic inverse problems illustrate our finding.

Bregman iterative regularization using model functions for nonconvex nonsmooth optimization

“…Assume that the limit point ξ * = 0. Noting that 1 δ k µ k ∇h(x k+1 ) − ∇h(x k ) → 0 and 1 µ k ξ k+1 → 1 µ ξ * = 0, we apply Lemma 4.8 in Zhang et al [7] to conclude that p 0 − p n+1 → ∞ as n → ∞, which contradicts the boundedness of {p k }. Therefore, we have ξ * = 0 ∈ ∂f (x * ).…”

“…The modified surrogate function is inspired by Benning et al [6] and Zhang et al [7]. However, their surrogate functions are invalid for our global convergence analysis, because the standard assumptions do not contain the subgradient relationship between the nonsmooth f and the model function.…”

“…Proof. We show Equation ( 10) by modifying the methodology in Zhang et al [7]. Let us begin with any point s * = (x * , x * , p * ) ∈ 0 .…”

“…Previous studies mainly focused on convex smooth optimization in the sense that both functions f and R in (P) are convex and f is also smooth. Very recently, nonconvex smooth extensions of LBI were considered in Benning et al [6] and later in Zhang et al [7]. However, it seems unclear whether the LBI can be extended to nonconvex and nonsmooth cases.…”

Alternating Block Linearized Bregman Iterations for Regularized Nonnegative Matrix Factorization

“…Definition 2.2 (Symmetric generalized Bregman distance [14]). D symm f (u, v) is called the symmetric generalized Bregman distance of f between u and v, if…”