Semi-supervised model-based document clustering: A comparative study

Abstract

Semi-supervised learning has become an attractive methodology for improving classification models and is often viewed as using unlabeled data to aid supervised learning. However, it can also be viewed as using labeled data to help clustering, namely, semisupervised clustering. Viewing semi-supervised learning from a clustering angle is useful in practical situations when the set of labels available in labeled data are not complete, i.e., unlabeled data contain new classes that are not present in labeled data. This paper analyzes several multinomial model-based semi-supervised document clustering methods under a principled model-based clustering framework. The framework naturally leads to a deterministic annealing extension of existing semi-supervised clustering approaches. We compare three (slightly) different semi-supervised approaches for clustering documents: Seeded damnl, Constrained damnl, and Feedback-based damnl, where damnl stands for multinomial model-based deterministic annealing algorithm. The first two are extensions of the seeded k-means and constrained k-means algorithms studied by Basu et al. (2002); the last one is motivated by Cohn et al. (2003). Through empirical experiments on text datasets, we show that: (a) deterministic annealing can often significantly improve the performance of semi-supervised clustering; (b) the constrained approach is the best when available labels are complete whereas the feedback-based approach excels when available labels are incomplete.

A LDA-Based Approach for Semi-Supervised Document Clustering

“…Cluster seeds are derived from the constraints to initialize the cluster centroids [7], [9]. In [11], a comparative study of investigating annealing process for varies model-based semi-supervised document clustering approaches with labeled documents are presented. Recently, Yan et al [12] investigated a semi-supervised fuzzy co-clustering approach.…”

“…Experimental Performance. For comparative investigation, two state-of-the-art semi-supervised document clustering approaches [11], [12] that use labeled documents as supervised information were investigated, labeled as constrained-DAMNL and SS-HFCR respectively, Fig. 7 shows the experimental performances of our proposed LLDA model, the constrained-DAMNL, and the SS-HFCR model on the re0 and the Yahoo_k1 datasets.…”

Absolute and relative clustering

“…If clusters consist of documents on the same topic, each topic in the fixed categorization corresponds to one cluster in an absolute partition. This type of document clustering is the absolute type [20]. In the second case, there are multiple kinds of topic categorizations, and documents are clustered based on one of the categorizations that would best appropriately summarize the document set.…”

“…Various types of problem formalizations as well as algorithms have been proposed as methods of supervised clustering. For example, users represent their preferences for grouping structures by labels [20]. Data points that are assigned the same label are grouped into the same cluster, and data with different labels are separated into different clusters.…”

A Semi-supervised Learning Framework to Cluster Mixed Data Types

“…Many semi-supervised algorithms have been proposed (Zhong, 2006) including co-training (Blum and Mitchell, 1998), the transductive support vector machine (Joachims, 1999), entropy minimization (Guerrero-Curieses and Cid-Sueiro, 2000), semi-supervised Expectation Maximization (Nigam et al, 2000), graph-based approaches (Blum and Chawla, 2001;Zhu et al, 2003), and clustering-based approaches (Zeng et al, 2003).…”