An ADMM Algorithm for Clustering Partially Observed Networks

Abstract

Community detection has attracted increasing attention during the past decade, and many algorithms have been proposed to find the underlying community structure in a given network. Many of these algorithms are based on modularity maximization, and these methods suffer from the resolution limit. In order to detect the underlying cluster structure, we propose a new convex formulation to decompose a partially observed adjacency matrix of a network into low-rank and sparse components. In such decomposition, the low-rank component encodes the cluster structure under certain assumptions. We also devise an alternating direction method of multipliers with increasing penalty sequence to solve this problem; and compare it with Louvain method, which maximizes the modularity, on some synthetic randomly generated networks. Numerical results show that our method outperforms Louvain method on the randomly generated networks when variance among cluster sizes increases. Moreover, empirical results also demonstrate that our formulation is indeed tighter than the robust PCA formulation, and is able to find the true clustering when the robust PCA formulation fails.

Convergence Rate of Distributed ADMM Over Networks

“…More recently, it has found applications in a variety of distributed settings in machine learning such as model fitting, resource allocation, and classification (see e.g. [36], [37], [38], [39], [40], [41], [42], [43], [44]).…”

Exterior-point Optimization for Nonconvex Learning

“…In [32], Wen et al propose an ADMM-based algorithm to solve the classical and ptychographic phase retrieval problems and empirically show that the algorithm outperforms convex relaxation based approaches. To solve clustering problems over partially observed networks, Aybat et al investigate the performance of an ADMM-based heuristic in [33] and show that under certain setups, their approach does better than the robust PCA method. Takapoui et al introduce an ADMM-based heuristic in [34] and its extension in [35] to solve mixed-integer quadratic programming with applications in control and maximum-likelihood decoding problems.…”

“…For notational convenience, we have used J γ∇f = prox γf , R γ∇f = J γ∇f − I and R γ µ ι = 2J γ µ ι − I. Then x µ ∈ B(x; r µ ) satisfies ⇔ (∃y ∈ E) x µ = J γ∇ µ ι R γ∇f (y) and x µ = J γ∇f (y), (33) where a) uses the facts that J γ∇f is a single-valued operator everywhere, whereas J γ∇ µ ι is a singlevalued operator on the region of convexity B(x; r µ ) (Lemma 4), and b) uses the observation that x µ = J γ∇f (y) can be expressed as…”

An alternating direction method with increasing penalty for stable principal component pursuit

“…(1.6) PCP and SPCP both have numerous applications in diverse fields such as video surveillance and face recognition in image processing [8], and clustering in machine learning [3] to name a few. (1.1), (1.4) and (1.6) can be reformulated as semidefinite programming (SDP) problems, and therefore, in theory they can be solved in polynomial time using interior point algorithms; however, these algorithms require very large amount of memory, and are, therefore, impractical for solving large instances.…”

“…In an earlier preprint, we named it as Non-Smooth Augmented Lagrangian (NSA) algorithm 3. The modified version is available from http://svt.stanford.edu/code.html…”