An efficient algorithm for sparse inverse covariance matrix estimation based on dual formulation

Li, Peili; Xiao, Yunhai

doi:10.1016/j.csda.2018.07.011

Cited by 16 publications

(15 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Denote J cT := (I + cT ) −1 , i.e., a resolvent operator, then the DRs [4] for solving (12) is with the form…”

Section: Preliminariesmentioning

confidence: 99%

“…In this section, we construct a series of numerical experiments using simulated convex composite optimization problems and sparse inverse covariance matrix estimation problems to demonstrate the feasibility and efficiency of the algorithm MGADMM. Besides, we also test against the state-of-the-art algorithm sPADMM of Li et al [11] and sGS-ADMM of Li & Xiao [12] to evaluate the algorithms' performance. All the experiments are performed with Microsoft Windows 10 and MATLAB R2018a, and run on a desktop PC with an Intel Core i7-8565 CPU at 1.80 GHz and 8 GB of memory.…”

Section: Numerical Experimentsmentioning

confidence: 99%

“…To implement the proposed algorithm, we also illustrate how to choose the positive semi-definite linear operators S and T for purpose of splitting each subproblem when the sGS technique is used. Finally, using some simulated convex composite optimization problems and some sparse inverse covariance matrix estimation problems, we do some numerical experiments versus the majorized ADMM of Li et al [11] and sGS-ADMM of Li & Xiao [12]. The results demonstrate that the variant of generalized-ADMM with a special relaxation factor generally performs better.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Majorized-Generalized Alternating Direction Method of Multipliers for Convex Composite Programming

Qin¹,

Xiao²,

Li³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The linearly constrained convex composite programming problems whose objective function contains two blocks with each block being the form of nonsmooth+smooth arises frequently in multiple fields of applications. If both of the smooth terms are quadratic, this problem can be solved efficiently by using the symmetric Gaussian-Seidel (sGS) technique based proximal alternating direction method of multipliers (ADMM). However, in the non-quadratic case, the sGS technique can not be used any more, which leads to the separable structure of nonsmooth+smooth had to be ignored. In this paper, we present a generalized ADMM and particularly use a majorization technique to make the corresponding subproblems more amenable to efficient computations. Under some appropriate conditions, we prove its global convergence for the relaxation factor in (0, 2). We apply the algorithm to solve a kind of simulated convex composite optimization problems and a type of sparse inverse covariance matrix estimation problems which illustrates that the effectiveness of the algorithm are obvious.

show abstract

“…Denote J cT := (I + cT ) −1 , i.e., a resolvent operator, then the DRs [4] for solving (12) is with the form…”

Section: Preliminariesmentioning

confidence: 99%

Section: Numerical Experimentsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Majorized-Generalized Alternating Direction Method of Multipliers for Convex Composite Programming

Qin¹,

Xiao²,

Li³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Wang [17] first generated an initial point using the proximal augmented Lagrangian method, then applied the Newton-CG augmented Lagrangian method to problems with an additional convex quadratic term in (P). Li and Xiao [13] employed the symmetric Gauss-Seidel-type ADMM in the same framework of [18]. A more recent work by Zhang et al [21] shows that (P) with simple constraints as X ij = 0 for (i, j) ∈ Ω can be converted into a more computationally tractable problem for large values of ρ.…”

Section: Introductionmentioning

confidence: 99%

A dual spectral projected gradient method for log-determinant semidefinite problems

Nakagaki

Fukuda

Kim³

et al. 2020

Comput Optim Appl

View full text Add to dashboard Cite

We extend the result on the spectral projected gradient method by Birgin et al. in 2000 to a log-determinant semidefinite problem (SDP) with linear constraints and propose a spectral projected gradient method for the dual problem. Our method is based on alternate projections on the intersection of two convex sets, which first projects onto the box constraints and then onto a set defined by a linear matrix inequality. By exploiting structures of the two projections, we show the same convergence properties can be obtained for the proposed method as Birgin's method where the exact orthogonal projection onto the intersection of two convex sets is performed. Using the convergence properties, we prove that the proposed algorithm attains the optimal value or terminates in a finite number of iterations. The efficiency of the proposed method is illustrated with the numerical results on randomly generated synthetic/deterministic data and gene expression data, in comparison with other methods including the inexact primaldual path-following interior-point method, the adaptive spectral projected gradient method, and the adaptive Nesterov's smooth method. For the gene expression data, our results are compared with the quadratic approximation for sparse inverse covariance estimation method. We show that our method outperforms the other methods in obtaining a better optimal value fast.Key words. Dual spectral projected gradient methods, log-determinant semidefinite programs with linear constraints, dual problem, theoretical convergence results, computational efficiency. AMS Classification. 90C20, 90C22, 90C25, 90C26.

show abstract

“…As shown in [2] that model (1.3) has the potential ability to discover the underlying distribution's structure when possible noises are contained on the sample covariance matrix S. In many applications, block-wise sparsity structure in the inverse covariance matrix is highly desirable. Because of this, many authors [10,14,17,19] considered the following general estimation model based on the adaptive group lasso…”

Section: Introductionmentioning

confidence: 99%

Semi-proximal augmented Lagrangian method for sparse estimation of high-dimensional inverse covariance matrices

2020

J. Appl. Numer. Optim.

View full text Add to dashboard Cite

Estimating a large and sparse inverse covariance matrix is a fundamental problem in modern multivariate analysis. Recently, a generalized model for a sparse estimation was proposed in which an explicit eigenvalue bounded constraint is involved. It covers a large number of existing estimation approaches as special cases. It was shown that the dual of the generalized model contains five separable blocks, which cause more challenges for minimizing. In this paper, we use an augmented Lagrangian method to solve the dual problem, but we minimize the augmented Lagrangian function with respect to each variable in a Jacobian manner, and add a proximal point term to make each subproblem easy to solve. We show that this iterative scheme is equivalent to adding a proximal point term to the augmented Lagrangian function, and its convergence can be directly followed. Finally, we give numerical simulations by using the synthetic data which show that the proposed algorithm is very effective in estimating high-dimensional sparse inverse covariance matrices.

show abstract

An efficient algorithm for sparse inverse covariance matrix estimation based on dual formulation

Cited by 16 publications

References 37 publications

A Majorized-Generalized Alternating Direction Method of Multipliers for Convex Composite Programming

A Majorized-Generalized Alternating Direction Method of Multipliers for Convex Composite Programming

A dual spectral projected gradient method for log-determinant semidefinite problems

Semi-proximal augmented Lagrangian method for sparse estimation of high-dimensional inverse covariance matrices

Contact Info

Product

Resources

About