Incorporating word embeddings into topic modeling of short text

Gao, Wang; Peng, Min; Wang, Hua; Zhang, Yanchun; Xie, Qianqian; Tian, Gang

doi:10.1007/s10115-018-1314-7

Cited by 58 publications

(30 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, researchers have improved this model. Gao et al [15] introduce word embedding information and conditional random fields in the topic model to model short text topics. Lu et al [16] introduce recurrent neural networks to identify the relationships between words based on the BTM model.…”

Section: A the Public Opinion Topic Identification Methods Based On mentioning

confidence: 99%

ST-ETM: A Spatial-Temporal Emergency Topic Model for Public Opinion Identifying in Social Networks

2020

View full text Add to dashboard Cite

With the development and popularization of social networks, many users and authoritative media broach and share topics through social networks every day. This type of sharing is widespread and is sometimes the first communication of topics of public discussion. Therefore, research on identifying public opinion topics on social networks is of great significance. However, social network posts are short, sparse and noisy, which creates challenges for this research. To overcome these challenges, we propose a spatialtemporal emergency topic model (ST-ETM) to identify public opinion topics in social networks. By introducing spatial-temporal features into the topic model to construct spatial-temporal regions for focusing on topics, the ST-ETM can alleviate the sparsity problem of context in social networks and succeed in focusing on public opinion topics. Moreover, to automatically identify public opinion topics, we introduce the burstiness of the words as a priori in the model, and binary switch variables are combined to automatically identify public opinion topics in social networks. Based on a real Sina Weibo dataset, several comparative experiments are designed to evaluate the performance of our ST-ETM. The experimental results verify the effectiveness of the proposed ST-ETM method. INDEX TERMS public opinion topic, social network, topic model, topic identification.

show abstract

Section: A the Public Opinion Topic Identification Methods Based On mentioning

confidence: 99%

ST-ETM: A Spatial-Temporal Emergency Topic Model for Public Opinion Identifying in Social Networks

2020

View full text Add to dashboard Cite

show abstract

“…They considered only contextual word-occurrence relationships based on the skipgram model, resulting in latent topics represented by bi-gram distributions and word embeddings per topic, which significantly increased the size of topic representations and extra computation cost. Gao et al [36] designed a Conditional Random Field regularized Topic Model (CRFTM), which assigns semantically related words to the same topic more probably. Word embeddings serve as the similarity measure to quantify the semantic correlations among words.…”

Section: Latent Topic Models Utilizing Word Embeddingsmentioning

confidence: 99%

Collaboratively Modeling and Embedding of Latent Topics for Short Texts

et al. 2020

View full text Add to dashboard Cite

Deriving a successful document representation is the critical challenge in many downstream tasks in NLP, especially when documents are very short. It is challenging to handle the sparsity and the noise problems confronting short texts. Some approaches employ latent topic models, based on global word co-occurrence, to obtain topic distribution as the representation. Others leverage word embeddings, which consider local conditional dependencies, to map a document as a summation vector of them. Unlike the existing works which explore the strategy of utilizing one to help the other, i.e., topic models for word embeddings or vice versa, we propose CME-DMM, a collaboratively modeling and embedding framework for capturing coherent latent topics from short texts. CME-DMM incorporates topic and word embeddings through the attention mechanism and implants them into the latent topic models, which significantly improve the quality of latent topics. Extensive experiments demonstrate that CME-DMM could perceive more coherent topics than other popular methods, resulting in a better performance in downstream NLP tasks such as classification. Besides the interpretable latent topics, the corresponding topic embeddings can describe the meanings of latent topics in the semantic space. The attention vectors, as a by-product of the learning process, can identify the keywords in noisy short texts.

show abstract

“…Topic modeling approaches working on short texts from social media platforms suffer from data sparsity, noisy words and word sense disambiguation problems. Gao et al [26] addressed the issue of word sense disambiguation by utilizing local and global semantic correlation provided by word embedding model. Conditional random field is used in inference phase for short text modeling.…”

Section: Related Workmentioning

confidence: 99%

Adaptive Model for Dynamic and Temporal Topic Modeling from Big Data using Deep Learning Architecture

Pathak¹,

Pandey²,

Rautaray³

2019

IJISA

View full text Add to dashboard Cite

Due to freedom to express views, opinions, news, etc and easier method to disseminate the information to large population worldwide, social media platforms are inundated with big streaming data characterized by both short text and long normal text. Getting the glimpse of ongoing events happening over social media is quintessential from the viewpoint of understanding the trends, and for this, topic modeling is the most important step. With reference to increase in proliferation of big data streaming from social media platforms, it is crucial to perform large scale topic modeling to extract the topics dynamically in an online manner. This paper proposes an adaptive framework for dynamic topic modeling from big data using deep learning approach. Approach based on approximation of online latent semantic indexing constrained by regularization has been put forth. The model is designed using deep network of feed forward layers. The framework works in an adaptive manner in the sense that model is extracts incrementally according to streaming data and retrieves dynamic topics. In order to get the trends and evolution of topics, the framework supports temporal topic modeling, and enables to detect implicit and explicit aspects from sentences also. regularization.  The approach is scalable to large collection of datasets. It is flexible to support both long normal text and short text for modeling the topics.  The model is adaptive and it is updated incrementally and performs temporal topic modeling to get notion of evolution and trends of topics over time.  It supports extraction of implicit and explicit topics from sentences also.

show abstract

Incorporating word embeddings into topic modeling of short text

Cited by 58 publications

References 26 publications

ST-ETM: A Spatial-Temporal Emergency Topic Model for Public Opinion Identifying in Social Networks

ST-ETM: A Spatial-Temporal Emergency Topic Model for Public Opinion Identifying in Social Networks

Collaboratively Modeling and Embedding of Latent Topics for Short Texts

Adaptive Model for Dynamic and Temporal Topic Modeling from Big Data using Deep Learning Architecture

Contact Info

Product

Resources

About