Enhanced proximal DC algorithms with extrapolation for a class of structured nonsmooth DC minimization

Lu, Zhaosong; Zhou, Zirui; Sun, Zhe

doi:10.1007/s10107-018-1318-9

Cited by 37 publications

(25 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to verify this property, we essentially assume that ϕ ↑ , ϕ ↓ , ψ 1 , and ψ 2 are differentiable with locally Lipschitz continuous gradients near Z ∞ . This type of assumptions have also been made in [32,57,36] to prove the sequential convergence of the dc algorithm (with extrapolation) for solving the dc program: minimize…”

Section: 1mentioning

confidence: 99%

Composite Difference-Max Programs for Modern Statistical Estimation Problems

Cui¹,

Pang²,

Sen³

2018

SIAM J. Optim.

View full text Add to dashboard Cite

Many modern statistical estimation problems are defined by three major components: a statistical model that postulates the dependence of an output variable on the input features; a loss function measuring the error between the observed output and the model predicted output; and a regularizer that controls the overfitting and/or variable selection in the model. We study the sampling version of this generic statistical estimation problem where the model parameters are estimated by empirical risk minimization, which involves the minimization of the empirical average of the loss function at the data points weighted by the model regularizer. In our setup we allow all three component functions discussed above to be of the difference-of-convex (dc) type and illustrate them with a host of commonly used examples, including those in continuous piecewise affine regression and in deep learning (where the activation functions are piecewise affine). We describe a nonmonotone majorization-minimization (MM) algorithm for solving the unified nonconvex, nondifferentiable optimization problem which is formulated as a specially structured composite dc program of the pointwise max type, and present convergence results to a directional stationary solution. An efficient semismooth Newton method is proposed to solve the dual of the MM subproblems. Numerical results are presented to demonstrate the effectiveness of the proposed algorithm and the superiority of continuous piecewise affine regression over the standard linear model.

show abstract

Section: 1mentioning

confidence: 99%

Composite Difference-Max Programs for Modern Statistical Estimation Problems

Cui¹,

Pang²,

Sen³

2018

SIAM J. Optim.

View full text Add to dashboard Cite

show abstract

“…algorithm has been further developed for improving the quality of solutions and the rate of convergence [30,31,36,48]. Most existing DC algorithms were proved to be subsequential convergent to a critical point of DC minimization for the case that g is nonsmooth [2,17,48].…”

Section: Introductionmentioning

confidence: 99%

“…When the subtracted function is defined by g = max 1≤i≤I ψ i (x) with convex and continuously differentiable ψ i , by exploiting the structure of the subtracted function in DC minimization, Pang, Razaviyayn, and Alvarado [36] proposed an enhanced DC algorithm to solve DC minimization with subsequential convergence to a d-stationary point of the considered problem. Further, Lu, Zhou and Sun [30,31] combined the enhanced DC algorithm with some possible accelerated methods to design some DC algorithms with subsequential convergence to the d-stationary points [30,31]. In problem (2), the subtracted part is the maximum of some convex functions.…”

Section: Introductionmentioning

confidence: 99%

“…In problem (2), the subtracted part is the maximum of some convex functions. But based on the nondifferentiablity of 1 -norm and 2 -norm, the DC algorithms in [30,31,36] cannot be used to solve the problem (2). Considering the special structure of the subtracted function in (2) and inspired by the idea in [30,31,36], we will explore the structure of the relaxation function to design two DC algorithms and get a stationary point satisfying a stronger optimality condition than the critical points for problem (2).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

DC algorithms for a class of sparse group $\ell_0$ regularized optimization problems

Li¹,

Bian²,

Toh³

2021

Preprint

View full text Add to dashboard Cite

In this paper, we consider a class of sparse group 0 regularized optimization problems. Firstly, we give a continuous relaxation model of the considered problem and establish the equivalence of these two problems in the sense of global minimizers. Then, we define a class of stationary points of the relaxation problem, and prove that any defined stationary point is a local minimizer of the considered sparse group 0 regularized problem and satisfies a desirable property of its global minimizers. Further, based on the difference-of-convex (DC) structure of the relaxation problem, we design two DC algorithms to solve the relaxation problem. We prove that any accumulation point of the iterates generated by them is a stationary point of the relaxation problem. In particular, all accumulation points have a common support set and a unified lower bound for the nonzero entries, and their zero entries can be attained within finite iterations. Moreover, we prove the convergence of the entire iterates generated by the proposed algorithms. Finally, we give some numerical experiments to show the efficiency of the proposed algorithms.

show abstract

“…Difference-of-convex (DC) programs are a class of important optimization problems, which generally minimize an objective function that is the difference of two convex functions subject to constraints defined by the same type of functions. They have been studied for several decades in the literature (e.g., see [10,17,11,25,28,20,16,13] and references therein). In this paper we are interested in a DC program in the form of min x∈X F (x) = φ 0 (x) + ζ 0 (x) − ψ 0 (x) s.t.…”

Section: Introductionmentioning

confidence: 99%

Penalty and Augmented Lagrangian Methods for Constrained DC Programming

Lu¹,

Sun²,

Shi³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper we consider a class of structured nonsmooth difference-of-convex (DC) constrained DC program in which the first convex component of the objective and constraints is the sum of a smooth and nonsmooth functions while their second convex component is the supremum of finitely many convex smooth functions. The existing methods for this problem usually have a weak convergence guarantee or require a feasible initial point. Inspired by the recent work (Math Oper. Res. 42(1):95-118, 2017 by Pang et al.), in this paper we propose two infeasible methods with strong convergence guarantee for the considered problem. The first one is a penalty method that consists of finding an approximate D-stationary point of a sequence of penalty subproblems. We show that any feasible accumulation point of the solution sequence generated by such a penalty method is a B-stationary point of the problem under a weakest possible assumption that it satisfies a pointwise Slater constraint qualification (PSCQ). The second one is an augmented Lagrangian (AL) method that consists of finding an approximate D-stationary point of a sequence of AL subproblems. Under the same PSCQ condition as for the penalty method, we show that any feasible accumulation point of the solution sequence generated by such an AL method is a B-stationary point of the problem, and moreover, it satisfies a KKT type of optimality condition for the problem, together with any accumulation point of the sequence of a set of auxiliary Lagrangian multipliers. We also propose an efficient successive convex approximation method for computing an approximate D-stationary point of the penalty and AL subproblems. Finally, some numerical experiments are conducted to demonstrate the efficiency of our proposed methods.

show abstract

Enhanced proximal DC algorithms with extrapolation for a class of structured nonsmooth DC minimization

Cited by 37 publications

References 16 publications

Composite Difference-Max Programs for Modern Statistical Estimation Problems

Composite Difference-Max Programs for Modern Statistical Estimation Problems

DC algorithms for a class of sparse group $\ell_0$ regularized optimization problems

Penalty and Augmented Lagrangian Methods for Constrained DC Programming

Contact Info

Product

Resources

About