Incorporating domain knowledge into topic modeling via Dirichlet Forest priors

Abstract

Users of topic modeling methods often have knowledge about the composition of words that should have high or low probability in various topics. We incorporate such domain knowledge using a novel Dirichlet Forest prior in a Latent Dirichlet Allocation framework. The prior is a mixture of Dirichlet tree distributions with special structures. We present its construction, and inference via collapsed Gibbs sampling. Experiments on synthetic and real datasets demonstrate our model’s ability to follow and generalize beyond user-specified domain knowledge.

Probabilistic Models for Text Mining

“…Another sort of guidance is to specify which terms should have similar probabilities in one topic (must-link) and which terms should not have similar probabilities in any topic (cannot-link). This kind of prior can be modeled as Dirichlet forest prior, which is discussed in [4].…”

Rapid Exploitation and Analysis of Documents

“…While these models were not found to be helpful for document smoothing [124], rich hierarchical topics may be beneficial when combined with the explicit user feedback present in our approach. Our approach could also exploit prior information such as predefined concepts by using topic model variants which can incorporate domain knowledge [24,5].…”

Constrained LDA for Grouping Product Features in Opinion Mining

“…To the best of our knowledge, this is the first constrained LDA model which can process large scale constraints in the forms of must-links and cannot-links. There are two existing work by Andrzejewski and Zhu [1,2] that are related to the proposed model. However, [1] only considers must-link constraints.…”

“…However, [1] only considers must-link constraints. In [2], the number of maximal cliques grow exponentially in the process of encoding constraints. Thus, [2] cannot process a large number of constraints (see Section 2.1).…”

“…In [2], the number of maximal cliques grow exponentially in the process of encoding constraints. Thus, [2] cannot process a large number of constraints (see Section 2.1). As we will also see in Section 2, our method of incorporating the two types of constraints is entirely different from the way that they did.…”