Abstract:We propose a proximal augmented Lagrangian method and a hybrid method, i.e., employing the proximal augmented Lagrangian method to generate a good initial point and then employing the Newton-CG augmented Lagrangian method to get a highly accurate solution, to solve large-scale nonlinear semidefinite programming problems whose objective functions are a sum of a convex quadratic function and a log-determinant term. We demonstrate that the algorithms can supply a high quality solution efficiently even for some il… Show more
“…Accordingly, many algorithms for solving optimization problems including the logdet function have been studied extensively so far. For example, see [16,17,18].…”
In this paper, we consider a nonlinear semi-infinite program that minimizes a function including a log-determinant (logdet) function over positive definite matrix constraints and infinitely many convex inequality constraints, called SIPLOG for short. The main purpose of the paper is to develop an algorithm for computing a Karush-Kuhn-Tucker (KKT) point for the SIPLOG efficiently. More specifically, we propose an interior point sequential quadratic programming-type method that inexactly solves a sequence of semi-infinite quadratic programs approximating the SIPLOG. Furthermore, to generate a search direction in the dual matrix space associated with the semi-definite constraint, we solve scaled Newton equations that yield the family of Monteiro-Zhang directions. We prove that the proposed method weakly* converges to a KKT point under some mild assumptions. Finally, we conduct some numerical experiments to demonstrate the efficiency of the proposed method.Keyword: semi-infinite program, log-determinant, nonlinear semi-definite program, sequential quadratic programming method, exchange method
“…Accordingly, many algorithms for solving optimization problems including the logdet function have been studied extensively so far. For example, see [16,17,18].…”
In this paper, we consider a nonlinear semi-infinite program that minimizes a function including a log-determinant (logdet) function over positive definite matrix constraints and infinitely many convex inequality constraints, called SIPLOG for short. The main purpose of the paper is to develop an algorithm for computing a Karush-Kuhn-Tucker (KKT) point for the SIPLOG efficiently. More specifically, we propose an interior point sequential quadratic programming-type method that inexactly solves a sequence of semi-infinite quadratic programs approximating the SIPLOG. Furthermore, to generate a search direction in the dual matrix space associated with the semi-definite constraint, we solve scaled Newton equations that yield the family of Monteiro-Zhang directions. We prove that the proposed method weakly* converges to a KKT point under some mild assumptions. Finally, we conduct some numerical experiments to demonstrate the efficiency of the proposed method.Keyword: semi-infinite program, log-determinant, nonlinear semi-definite program, sequential quadratic programming method, exchange method
“…They demonstrated that regularization using • 2 or • ∞ norms instead of • 1 in (P) are more suitable for the structured models/problems. Wang [17] first generated an initial point using the proximal augmented Lagrangian method, then applied the Newton-CG augmented Lagrangian method to problems with an additional convex quadratic term in (P). Li and Xiao [13] employed the symmetric Gauss-Seidel-type ADMM in the same framework of [18].…”
Section: Introductionmentioning
confidence: 99%
“…A more recent work by Zhang et al [21] shows that (P) with simple constraints as X ij = 0 for (i, j) ∈ Ω can be converted into a more computationally tractable problem for large values of ρ. Among the methods mentioned here, only the methods discussed in [18,19,17] can handle problems as general as (P).…”
We extend the result on the spectral projected gradient method by Birgin et al. in 2000 to a log-determinant semidefinite problem (SDP) with linear constraints and propose a spectral projected gradient method for the dual problem. Our method is based on alternate projections on the intersection of two convex sets, which first projects onto the box constraints and then onto a set defined by a linear matrix inequality. By exploiting structures of the two projections, we show the same convergence properties can be obtained for the proposed method as Birgin's method where the exact orthogonal projection onto the intersection of two convex sets is performed. Using the convergence properties, we prove that the proposed algorithm attains the optimal value or terminates in a finite number of iterations. The efficiency of the proposed method is illustrated with the numerical results on randomly generated synthetic/deterministic data and gene expression data, in comparison with other methods including the inexact primaldual path-following interior-point method, the adaptive spectral projected gradient method, and the adaptive Nesterov's smooth method. For the gene expression data, our results are compared with the quadratic approximation for sparse inverse covariance estimation method. We show that our method outperforms the other methods in obtaining a better optimal value fast.Key words. Dual spectral projected gradient methods, log-determinant semidefinite programs with linear constraints, dual problem, theoretical convergence results, computational efficiency. AMS Classification. 90C20, 90C22, 90C25, 90C26.
“…Moreover, below we introduce two standard assumptions for the log-determinant problem [37] and Gibbs sampler [54] respectively. Assumption 4.2.…”
Section: Theoretical Resultsmentioning
confidence: 99%
“…The augmented Lagrangian method is exploited in [37] to solve the log-determinant optimization. It combines the proximal augmented Lagrangian and Newton-CG augmented Lagrangian as a hybrid approach.…”
Graphical models compactly represent the most significant interactions of multivariate probability distributions, provide an efficient inference framework to answer challenging statistical queries, and incorporate both expert knowledge with data to extract information from complex systems. When the graphical model is assumed to be Gaussian, the resulting model features attractive properties and appears frequently in cutting-edge applications. In the application of Gaussian graphical models, it is fundamental and challenging to learn the graph structure from given multivariate data. The time complexity of existing methods is subject to the cumbersome operations such as matrix inversion, full spectral decomposition, and Cholesky decompositions, which generally take O(N 3 ) arithmetic operations. This computational bottleneck severely impedes the scalability of those methods into problems at large scale. Moreover, structure learning in Gaussian graphical model is a specialized case of the general log-determinant optimization, which entails general matrix constraints and covers much broader fields of applications. The bottleneck of the cubic time complexity also occurs in existing methods for the general log-determinant optimization. In this thesis, we propose a novel method to learn the structure of the Gaussian graphical model with the aid of stochastic approximation so that the time complexity can be reduced from O(N 3 ) to O(N 2 ). In addition, we generalize the proposed method within the framework of primal-dual optimization so that we can settle the general log-determinant optimization problem with similar efficiency.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.