Latent tree models for hierarchical topic detection

Abstract

We present a novel method for hierarchical topic detection where topics are obtained by clustering documents in multiple ways. Specifically, we model document collections using a class of graphical models called hierarchical latent tree models (HLTMs). The variables at the bottom level of an HLTM are observed binary variables that represent the presence/absence of words in a document. The variables at other levels are binary latent variables, with those at the lowest latent level representing word co-occurrence patterns and those at higher levels representing co-occurrence of patterns at the level below. Each latent variable gives a soft partition of the documents, and document clusters in the partitions are interpreted as topics. Latent variables at high levels of the hierarchy capture long-range word co-occurrence patterns and hence give thematically more general topics, while those at low levels of the hierarchy capture short-range word co-occurrence patterns and give thematically more specific topics. Unlike LDA-based topic models, HLTMs do not refer to a document generation process and use word variables instead of token variables. They use a tree structure to model the relationships between topics and words, which is conducive to the discovery of meaningful topics and topic hierarchies.

A Semantic Metadata Enrichment Software Ecosystem Based on Topic Metadata Enrichments

“…In order to discover a set of strongly correlated topics, they used the CPM-based community detection algorithm to find groups of topics with strong correlations. As in [8], their contribution was limited to simulating existing algorithms. Zhang et al [15] proposed LDA-IG, an extension of KeyGraph [14].…”

“…This is carried out by the BRIDGEISLANDS subroutine, which estimates the mutual information between each pair of latent variables in the islands. This allows construction of a complete undirected graph with the mutual information values as edge weights, and finally the maximum spanning tree of the graph is determined [8]. Hurtado et al [18] proposed an approach that uses sentence-level association rule mining to discover topics from documents.…”

“…The literature presents two groups of text-based topic detection approaches based on the size of the text: short text [11,7,12,13] such as tweets or Facebook posts, and long text [14,4,[15][16][17]8] such as a document or a book. For example, Dang et al [11] proposed an early detection method for emerging topics based on dynamic Bayesian networks in micro-blogging networks.…”

“…While this is a satisfactory approach to find latent co-occurrence relations, the approach assumes that keywords are topics. Chen et al [8] proposed a novel method for hierarchical topic detection where topics are obtained by clustering documents in multiple ways. They used a class of graphical models called hierarchical latent tree models (HLTMs).…”

“…A number of approaches are used to perform text mining, including: latent Dirichlet allocation (LDA) [4], tf-idf [2,3], latent semantic analysis (LSA) [6], formal concept analysis (FCA) [7], latent tree model (LTM) [8], naïve Bayes (NB) [9], and artificial neural network (ANN) [10]. This paper consists of a model and an algorithm SATD (Scalable Annotation-based Topic Detection) for topic metadata semantic enrichments.…”