Low-rank quadratic semidefinite programming

Yuan, Ganzhao; Zhang, Zhenjie; Ghanem, Bernard; Zhang, Hao

doi:10.1016/j.neucom.2012.10.014

Cited by 3 publications

(5 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In every iteration, one can pick the coordinate of the largest real value in the descent direction −∇f ( w) and greedily decrease the objective function. Similar to the low-rank factorization M = LL T , one can employ the non-negative representation w = v ⊙ v and refine v using some efficient non-convex optimization approaches [2]. 7.…”

Section: Connection With Shalev-shwartz Et Al's Algorithm: Shalev-shmentioning

confidence: 99%

“…We use BILGO-KTA and BILGO-LMNN to denote the BILGO solver for the two metric learning optimization problems respectively. Moreover, we use "L-bfgs + exact line search" as it is suggested in [2] to improve the intermediate result in every 5 iterations of BILGO-KTA, giving rise to its local search version BILGO-KTA-LS.…”

Section: Accuracy and Efficiency On Metric Learningmentioning

confidence: 99%

“…For simplicity, we use π and τ instead of α and β in this proof. They are transformable by Eq (2). By Lemma 3, we have:…”

mentioning

confidence: 94%

“…Semidefinite Programming (SDP) is commonly used in machine learning problems, such as metric learning [1,2], manifold learning [3,4,5], linear regression [6], distance matrix completion [7], and solving Lyapunov equations [8]. Given a convex and continuously differentiable function F , where F : R d×d → R, the goal of general semidefinite programming is to find a positive semidefinite (PSD) matrix M such that the objective F (M) is minimized, as shown in Eq (1).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

BILGO: Bilateral greedy optimization for large scale semidefinite programming

2014

Self Cite

View full text Add to dashboard Cite

Many machine learning tasks (e.g. metric and manifold learning problems) can be formulated as convex semidefinite programs. To enable the application of these tasks on a large-scale, scalability and computational efficiency are considered desirable properties for a practical semidefinite programming algorithm. In this paper, we theoretically analyse a new bilateral greedy optimization(denoted BILGO) strategy in solving general semidefinite programs on large-scale datasets. As compared to existing methods, BILGO employs a bilateral search strategy during each optimization iteration. In such an iteration, the current semidefinite matrix solution is updated as a bilateral linear combination of the previous solution and a suitable rank-1 matrix, which can be efficiently computed from the leading eigenvector of the descent direction at this iteration. By optimizing for the coefficients of the bilateral combination, BILGO reduces the cost function in every iteration until the KKT conditions are fully satisfied, thus, it tends to converge to a global optimum. For an ǫ-accurate solution, we prove the number of BILGO iterations needed for convergence is O(ǫ −1). The algorithm thus successfully combines the efficiency of conventional rank-1 update algorithms and the effectiveness of gradient descent. Moreover, BILGO can be easily extended to handle low rank constraints. To validate the effectiveness and efficiency of BILGO, we apply it to two important machine learning tasks, namely Mahalanobis metric learning and maximum variance unfolding. Extensive experimental results clearly demonstrate that BILGO can solve large-scale semidefinite

show abstract

Section: Connection With Shalev-shwartz Et Al's Algorithm: Shalev-shmentioning

confidence: 99%

Section: Accuracy and Efficiency On Metric Learningmentioning

confidence: 99%

“…For simplicity, we use π and τ instead of α and β in this proof. They are transformable by Eq (2). By Lemma 3, we have:…”

mentioning

confidence: 94%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

BILGO: Bilateral greedy optimization for large scale semidefinite programming

2014

Self Cite

View full text Add to dashboard Cite

show abstract

“…Meanwhile, there are several methods available for solving some special case of (1), including the inexact interior-point methods [4,5,9,10], the alternating direction methods [7,22,30], the quasi-Newton method [11], the inexact semi-smooth Newton-CG method [21], inexact accelerated proximal gradient (IAPG) method [20] and other methods [8,24].…”

Section: Introductionmentioning

confidence: 99%