Adaptive context trees and text clustering

Vert, Jean-Philippe

doi:10.1109/18.930925

Cited by 11 publications

(5 citation statements)

References 21 publications

(35 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The communities are differentiated in a strong sense and in a weak sense in [26]. It worths to notice that many clustering algorithms can be used to discover communities under different conditions if we regard community discovery as a kind of clustering process [32] [34].…”

Section: On Community Discovery and Relation Querymentioning

confidence: 99%

Communities and Emerging Semantics in Semantic Link Network: Discovery and Learning

Zhuge

2009

IEEE Trans. Knowl. Data Eng.

243

136

View full text Add to dashboard Cite

Abstract-TheWorld Wide Web provides plentiful contents for Web-based learning, but its hyperlink-based architecture connects Web resources for browsing freely rather than for effective learning. To support effective learning, an e-learning system should be able to discover and make use of the semantic communities and the emerging semantic relations in dynamic complex network of learning resources. Previous graph-based community discovery approaches are limited in ability to discover semantic communities. This paper firstly suggests the Semantic Link Network SLN, a loosely coupled semantic data model that can semantically link resources and derive out implicit semantic links according to a set of relational reasoning rules. By studying the intrinsic relationship between semantic communities and the semantic space of SLN, approaches to discovering reasoning-constraint, rule-constaint and classification-constraint semantic communities are proposed. Further, the approaches, principles and strategies for discovering emerging semantics in dynamic SLN are studied. An e-learning environment incorporating the proposed approaches, principles and strategies to support effective discovery and learning is suggested.

show abstract

Section: On Community Discovery and Relation Querymentioning

confidence: 99%

Communities and Emerging Semantics in Semantic Link Network: Discovery and Learning

Zhuge

2009

IEEE Trans. Knowl. Data Eng.

243

136

View full text Add to dashboard Cite

show abstract

“…The most suitable idea for our setting is the Kullback-Leibler (KL) divergence (see Equation (2)), which gives a notion of the distance between two distributions. The KL divergence is commonly used in several fields to measure the distance between two probability density functions (PDFs, as in, e.g., information theory [35], pattern recognition [31,17]).…”

Section: Comparing Two Distributionsmentioning

confidence: 99%

D-Search: an efficient and exact search algorithm for large distribution sets

2010

View full text Add to dashboard Cite

Distribution data naturally arise in countless domains, such as meteorology, biology, geology, industry and economics. However, relatively little attention has been paid to data mining for large distribution sets. Given n distributions of multiple categories and a query distribution Q, we want to find similar clouds (i.e., distributions), to discover patterns, rules and outlier clouds. For example, consider the numerical case of sales of items, where, for each item sold, we record the unit price and quantity; then, each customer is represented as a distribution of 2-d points (one for each item he/she bought). We want to find similar users, e.g., for market segmentation or anomaly/fraud detection. We propose to address this problem and present D-Search, which includes fast and effective algorithms for similarity search in large distribution datasets. Our main contributions are (1) approximate KL divergence, which can speed up cloud-similarity computations, (2) multi-step sequential scan, which efficiently prunes a significant number of search candidates and leads to a direct reduction in the search cost. We also introduce an extended version of D-Search : (3) time-series distribution mining, which finds similar subsequences in time-series distribution datasets. Extensive experiments on real multi-dimensional datasets show that our solution achieves a wall clock time up to 2,300 times faster than the naive implementation without sacrificing accuracy.

show abstract

“…The most suitable idea for our setting is the KL (Kullback-Leibler) divergence (see Equation (3.1)) which gives a notion of distance between two distributions. The KL divergence is commonly used in several fields, to measure the distance between two PDFs (probability density functions, as in, e.g., information theory [22], pattern recognition [19,10]).…”

Section: Related Workmentioning

confidence: 99%

Efficient Distribution Mining and Classification

Sakurai¹,

Li²,

Chong³

et al. 2008

Proceedings of the 2008 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

We define and solve the problem of "distribution classification", and, in general, "distribution mining". Given n distributions (i.e., clouds) of multi-dimensional points, we want to classify them into k classes, to find patterns, rules and out-lier clouds. For example, consider the 2-d case of sales of items, where, for each item sold, we record the unit price and quantity; then, each customer is represented as a distribution/cloud of 2-d points (one for each item he bought). We want to group similar users together, e.g., for market segmentation, anomaly/fraud detection.We propose D-Mine to achieve this goal. Our main contribution is Theorem 3.1, which shows how to use wavelets to speed up the cloud-similarity computations. Extensive experiments on both synthetic and real multidimensional data sets show that our method achieves up to 400 faster wall-clock time over the naive implementation, with comparable (and occasionally better) classification quality.

show abstract

Adaptive context trees and text clustering

Cited by 11 publications

References 21 publications

Communities and Emerging Semantics in Semantic Link Network: Discovery and Learning

Communities and Emerging Semantics in Semantic Link Network: Discovery and Learning

D-Search: an efficient and exact search algorithm for large distribution sets

Efficient Distribution Mining and Classification

Contact Info

Product

Resources

About