Supervised topic models with word order structure for document classification and retrieval learning

Abstract

One limitation of most existing probabilistic latent topic models for document classification is that the topic model itself does not consider useful side-information, namely, class labels of documents. Topic models, which in turn consider the side-information, popularly known as supervised topic models, do not consider the word order structure in documents. One of the motivations behind considering the word order structure is to capture the semantic fabric of the document. We investigate a low-dimensional latent topic model for document classification. Class label information and word order structure are integrated into a supervised topic model enabling a more effective interaction among such information for solving document classification. We derive a collapsed Gibbs sampler for our model. Likewise, supervised topic models with word order structure have not been explored in document retrieval learning. We propose a novel supervised topic model for document retrieval learning which can be regarded as a pointwise model for tackling the learning-to-rank task. Available relevance assessments and word order structure are integrated into the topic model itself. We conduct extensive experiments on several publicly available benchmark datasets, and show that our model improves upon the state-of-the-art models.

Word and Document Embedding with vMF-Mixture Priors on Context Word Vectors

“…We have adopted the same preprocessing strategy as for the categorization task, with the exception of OHSUMED, for which suitable LTR features are already given. For all other datasets we used the Terrier LTR framework 23 to generate the six standard LTR document features as described in (Jameel et al, 2015). The document vectors were then concatenated with these six features 24 .…”

Document Classification System According to the Degree of Semantic Link Expressed Fuzzy Function

“…Based on the frequency of occurrence of the index term, this scheme attempts to find the most likely category into which new documents are supposed to be categorized. [6][7] As for the representative methods in the statistic document classification technique, there are two main methods: (1) the method to use Bayesian probability and (2) the method to use vector similarity. In the Bayesian probability based method, probabilities that documents are classified to each category are estimated whenever the index terms extracted from an arbitrary document appear.…”

Exploring Urban Lifestyles Using a Nonparametric Temporal Graphical Model

“…We learn the parameters of the model using the training data (75%), and report the perplexity results on the held-out data (25%). For the parametric topic models, we use a tuning set to determine the number of topics following the tuning procedure described in [13]. Our objective is to compare how well our model has learned all parameters and how it performs in terms of its generalization ability.…”