User Based Aggregation for Biterm Topic Model

Abstract

Biterm Topic Model (BTM) is designed to model the generative process of the word co-occurrence patterns in short texts such as tweets. However, two aspects of BTM may restrict its performance: 1) user individualities are ignored to obtain the corpus level words co-occurrence patterns; and 2) the strong assumptions that two co-occurring words will be assigned the same topic label could not distinguish background words from topical words. In this paper, we propose Twitter-BTM model to address those issues by considering user level personalization in BTM. Firstly, we use user based biterms aggregation to learn user specific topic distribution. Secondly, each user's preference between background words and topical words is estimated by incorporating a background topic. Experiments on a large-scale real-world Twitter dataset show that Twitter-BTM outperforms several stateof-the-art baselines.

A Latent Concept Topic Model for Robust Topic Inference Using Word Embeddings

“…BTM directly models the generation of biterms (pairs of words) in the whole corpus. However, the assumption that pairs of cooccurring words should be assigned to the same topic might be too strong (Chen et al, 2015).…”

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

“…Recent work on learning user representations with multitask deep learning techniques (Li et al, 2015), suggests that learning a nonlinear mapping from observed views to the latent space can learn high quality user representations. One issue with GCCA is scalability: solving for G relies on an SVD of a large matrix that must be loaded into memory.…”

“…One related work was proposed by Zhang and Wang (2015), which employed bidirectional RN-N to learn patterns of relations from raw text data. Although bidirectional RNN has access to both past and future context information, the range of context is limited due to the vanishing gradient problem.…”

“…Another related work is SDP-LSTM model proposed by Yan et al (2015). This model leverages the shortest dependency path(SDP) between two nominals, then it picks up heterogeneous information along the SDP with LSTM units.…”

“…, h T ] that the LSTM layer produced, where T is the sentence length. The representation r of the sentence is formed by a weighted sum of these output vectors: (Zhang and Wang, 2015) WV (dim=300) + PI 82.5 SDP-LSTM WV (pretrained by word2vec) (dim=200), syntactic parse 82.4 (Yan et al, 2015) + POS + WordNet + grammar relation embeddings 83.7 BLSTM WV ) (dim=100) 82.7 + PF + POS + NER + WNSYN + DEP 84.3 BLSTM WV where…”