Convex Clustering via l  1 Fusion Penalization

Abstract

We study the large sample behaviour of a convex clustering framework, which minimizes the sample within cluster sum of squares under an l 1 fusion constraint on the cluster centroids. This recently proposed approach has been gaining in popularity; however, its asymptotic properties have remained mostly unknown. Our analysis is based on a novel representation of the sample clustering procedure as a sequence of cluster splits determined by a sequence of maximization problems. We use this representation to provide a simple and intuitive formulation for the population clustering procedure. We then demonstrate that the sample procedure consistently estimates its population analogue and we derive the corresponding rates of convergence. The proof conducts a careful simultaneous analysis of a collection of M-estimation problems, whose cardinality grows together with the sample size. On the basis of the new perspectives gained from the asymptotic investigation, we propose a key post-processing modification of the original clustering framework. We show, both theoretically and empirically, that the resulting approach can be successfully used to estimate the number of clusters in the population. Using simulated data, we compare the proposed method with existing number-of-clusters and modality assessment approaches and obtain encouraging results. We also demonstrate the applicability of our clustering method to the detection of cellular subpopulations in a single-cell virology study.

Feature screening in large scale cluster analysis

“…Our theoretical results are a significant extension of the univariate results in Radchenko and Mukherjee [43]. We rely herein on a more careful analysis of the empirical processes corresponding to the clustering trees constructed for each feature by the associated clustering procedure.…”

“…The convexity of the objective function in (1) has been exploited to develop algorithms for efficiently producing the path of solutions as a function of the penalty weight [31,32,43]. Clustering algorithms based on fusion penalization of this type have become very popular in large scale clustering [13,31,43,51,58] and regression analysis [8,37,47,48]. The entire path of solutions corresponding to the objective criterion (1) can be found by a simple merge algorithm in O(n ln n) operations.…”

Dynamic Visualization and Fast Computation for Convex Clustering via Algorithmic Regularization

“…give sufficient conditions for exact cluster recovery at a fixed value of λ ("sparsistency"). Like us, Radchenko and Mukherjee (2017) are interested in properties of the entire solution path and give conditions under which convex clustering solutions (1) asymptotically yield the true dendrogram.…”

“…To address these limitations, several authors have recently studied a convex formulation of clustering (Pelckmans et al, 2005;. This convex formulation guarantees global optimality of the clustering solution and allows analysis of its theoretical properties (Tan and Witten, 2015;Radchenko and Mukherjee, 2017;Chi and Steinerberger, 2018). Despite these advantages, convex clustering has not yet achieved widespread popularity, due to its computationally intensive nature and lack of dendrogram-based visualizations.…”

Splitting Methods For Convex Bi-Clustering And Co-Clustering

“…These convex re-formulations have many noteworthy advantages over classical methods, including improved statistical performance, analytical tractability, and efficient and scalable computation. In particular, a convex formulation of clustering [1]- [3] has sparked much recent research, as it provides theoretical and computational guarantees not available for classical clustering methods [4]- [6]. Convex clustering combines a squared Frobenius norm loss term, which encourages the estimated centroids to remain near the original data, with a convex fusion penalty, typically the q -norm of the row-wise differences, which shrinks the estimated centroids together, inducing clustering behavior in the solution:…”