Approximating K‐means‐type Clustering via Semidefinite Programming

Abstract

One of the fundamental clustering problems is to assign n points into k clusters based on the minimal sum-of-squares(MSSC), which is known to be NP-hard. In this paper, by using matrix arguments, we first model MSSC as a so-called 0-1 semidefinite programming (SDP). We show that our 0-1 SDP model provides an unified framework for several clustering approaches such as normalized k-cut and spectral clustering. Moreover, the 0-1 SDP model allows us to solve the underlying problem approximately via the relaxed linear and semidefinite programming.Secondly, we consider the issue of how to extract a feasible solution of the original MSSC model from the approximate solution of the relaxed SDP problem. By using principal component analysis, we develop a rounding procedure to construct a feasible partitioning from a solution of the relaxed problem. In our rounding procedure, we need to solve a k-means clustering problem in ℜ k−1 , which can be solved in O(n k 2 (k−1) ) time. In case of bi-clustering, the running time of our rounding procedure can be reduced to O(n log n). We show that our algorithm can provide a 2-approximate solution to the original problem. Promising numerical results based on our new method are reported.

Signal Processing and Optimization Tools for Conference Review and Session Assignment

“…Modifications of k-means to account for must-link/cannot-link constraints are discussed in [15], distance-type constraints on the cluster centers are discussed in [16], and lower-bound constraints on the number of points per cluster are discussed in [17]. As an alternative to alternating optimization-based k-means, approximation algorithms based on convex (semidefinite) optimization [18] are also known; see, e.g., [19] and the references therein.…”

A Convex Approach to K-Means Clustering and Image Segmentation

“…Every column of h is on the simplex, h is positive semidefinite, its trace and rank are K. We can then reformulate the problem as minimizing h, w over the set of such matrices. Linear programming and semidefinite programming convex relaxations of this problem have been proposed [33][34][35]. They are less efficient in practice than the convex formulation (12) [35].…”

Efficient global optimization for exponential family PCA and low-rank matrix factorization

“…The first equality holds because both sets of constraints bound the eigenvalues of the matrices to be either 0 or 1, with exactly k being 1 [27]. Unfortunately, neither of the first two sets of constraints is convex on M .…”