Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints

Guo, Ke; Han, Deren; Wu, Tingting

doi:10.1080/00207160.2016.1227432

Cited by 91 publications

(75 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1b), and maximum inverse of the penalty paramenter 1 /β in ADMM ( Fig. 1c); comparison between our bounds (blue plot) and [26] for DRS, [24] for PRS and [18,19,23,25,35] for ADMM. On the xaxis the ratio between hypoconvexity parameter σ and the Lipschitz modulus L of the gradient of the smooth function.…”

Section: 2mentioning

confidence: 86%

See 1 more Smart Citation

Douglas--Rachford Splitting and ADMM for Nonconvex Optimization: Tight Convergence Results

2020

View full text Add to dashboard Cite

Although originally designed and analyzed for convex problems, the alternating direction method of multipliers (ADMM) and its close relatives, Douglas-Rachford splitting (DRS) and Peaceman-Rachford splitting (PRS), have been observed to perform remarkably well when applied to certain classes of structured nonconvex optimization problems. However, partial global convergence results in the nonconvex setting have only recently emerged. In this paper we show how the Douglas-Rachford envelope (DRE), introduced in 2014, can be employed to unify and considerably simplify the theory for devising global convergence guarantees for ADMM, DRS and PRS applied to nonconvex problems under less restrictive conditions, larger prox-stepsizes and over-relaxation parameters than previously known. In fact, our bounds are tight whenever the over-relaxation parameter ranges in (0, 2]. The analysis of ADMM uses a universal primal equivalence with DRS that generalizes the known duality of the algorithms. (DRS)The case λ = 1 corresponds to the classical DRS, whereas for λ = 2 the scheme is also known as Peaceman-Rachford splitting (PRS). If s is a fixed point for the DR-iterationthat is, such that s + = s -then it can be easily seen that u satisfies the first-order necessary condition for optimality in problem (1.1). When both ϕ 1 and ϕ 2 are convex functions, the

show abstract

Section: 2mentioning

confidence: 86%

“…The problem addressed in [19] is fully covered by our analysis, as they consider ADMM for (1.2) where f is L-Lipschitz continuously differentiable and B is the identity matrix. Their bound β > 2L for the penalty parameter is more conservative than ours; in fact, the two coincide only in a worst-case scenario.…”

Section: 2mentioning

confidence: 99%

Douglas--Rachford Splitting and ADMM for Nonconvex Optimization: Tight Convergence Results

2020

View full text Add to dashboard Cite

show abstract

“…Recently a great deal of attention has been focused on using ADMM to solve nonconvex problems [26,27,28,29,30,31,32]. The work [26] studies the convergence of traditional ADMM under relatively strict assumptions.…”

Section: Related Workmentioning

confidence: 99%

“…• Weaker assumptions: Our assumptions are less restrictive in comparison with previous works on nonconvex ADMM (see, e.g., [26,27,28,30,31,32]). Specifically, we put much weaker restriction on function f (f i ) and matrix A (A i ).…”

Section: Contributionmentioning

confidence: 99%

See 1 more Smart Citation

Linearized ADMM for Nonconvex Nonsmooth Optimization With Convergence Analysis

Liu

Shen

2019

IEEE Access

View full text Add to dashboard Cite

Linearized alternating direction method of multipliers (ADMM) as an extension of ADMM has been widely used to solve linearly constrained problems in signal processing, machine leaning, communications, and many other fields. Despite its broad applications in nonconvex optimization, for a great number of nonconvex and nonsmooth objective functions, its theoretical convergence guarantee is still an open problem. In this paper, we propose a two-block linearized ADMM and a multi-block parallel linearized ADMM for problems with nonconvex and nonsmooth objectives. Mathematically, we present that the algorithms can converge for a broader class of objective functions under less strict assumptions compared with previous works. Furthermore, our proposed algorithm can update coupled variables in parallel and work for less restrictive nonconvex problems, where the traditional ADMM may have difficulties in solving subproblems.

show abstract

Local Sensitive Low Rank Matrix Approximation via Nonconvex Optimization

Bao

et al. 2017

Intelligent Computing Methodologies

View full text Add to dashboard Cite

Low-rank matrix approximation (LRMA) has been arisen in many applications, such as dynamic MRI, recommendation system and so on. The alternating direction method of multipliers (ADMM) has been designed for the nuclear norm regularized least squares problem or rank constrained form of LRMA with specialized measurement operators and shows a good performance. However, due to the lack of guarantees for the nonconvex ADMM, there are no ADMM algorithms designed directly for the rank constraint form of the general LRMA. Therefore, in this paper, we propose an ADMM-based algorithm for the reformulated rank constraint matrix approximation problem, which works well for noiseless and noisy case. Based on the Kurdyka-Lojasiewicz (KL) property, we prove that our proposed nonconvex ADMM algorithm globally converges to the optimal solution when the original LRMA has the unique optimal solution. And we discuss a specific case: the matrix completion problem. Numerical experiments show that spcialized for matrix completion, the proposed algorithm performs better when the sampling rate is really low in noisy case, which is the key in matrix completion.

show abstract

Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints

Cited by 91 publications

References 30 publications

Douglas--Rachford Splitting and ADMM for Nonconvex Optimization: Tight Convergence Results

Douglas--Rachford Splitting and ADMM for Nonconvex Optimization: Tight Convergence Results

Linearized ADMM for Nonconvex Nonsmooth Optimization With Convergence Analysis

Local Sensitive Low Rank Matrix Approximation via Nonconvex Optimization

Contact Info

Product

Resources

About