Improving biterm topic model with word embeddings

Huang, Jie; Peng, Min; Li, Pengwei; Hu, Zhiwei; Xu, Chao

doi:10.1007/s11280-020-00823-w

Cited by 18 publications

(13 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Topic knowledge is able to provide additional information for short text classification [ 17 ]. Zeng et al proposed a topic memory network that encodes topic representations and classifies documents by memory networks [ 43 ].…”

Section: Related Workmentioning

confidence: 99%

Identifying informative tweets during a pandemic via a topic-aware neural language model

Gao

Tao

et al. 2022

World Wide Web

View full text Add to dashboard Cite

Every epidemic affects the real lives of many people around the world and leads to terrible consequences. Recently, many tweets about the COVID-19 pandemic have been shared publicly on social media platforms. The analysis of these tweets is helpful for emergency response organizations to prioritize their tasks and make better decisions. However, most of these tweets are non-informative, which is a challenge for establishing an automated system to detect useful information in social media. Furthermore, existing methods ignore unlabeled data and topic background knowledge, which can provide additional semantic information. In this paper, we propose a novel Topic-Aware BERT (TABERT) model to solve the above challenges. TABERT first leverages a topic model to extract the latent topics of tweets. Secondly, a flexible framework is used to combine topic information with the output of BERT. Finally, we adopt adversarial training to achieve semi-supervised learning, and a large amount of unlabeled data can be used to improve inner representations of the model. Experimental results on the dataset of COVID-19 English tweets show that our model outperforms classic and state-of-the-art baselines.

show abstract

Section: Related Workmentioning

confidence: 99%

Identifying informative tweets during a pandemic via a topic-aware neural language model

Gao

Tao

et al. 2022

World Wide Web

View full text Add to dashboard Cite

show abstract

“…Wu et al [59] introduced a clustering method for short texts based on the (BG & SLF-Kmeans) method. In addition, a novel approach named Noise BTM Word Embedding (NBTMWE) was suggested by [60] to resolve the data sparsity problems. This approach integrates the noise BTM and WE from external datasets to ameliorate the coherence of the topic.…”

Section: Short Text Topic Modellingmentioning

confidence: 99%

Enhancing Big Social Media Data Quality for Use in Short-Text Topic Modeling

et al. 2022

View full text Add to dashboard Cite

With the emergence of microblogging platforms and social media applications, large amounts of user-generated data in the form of comments, reviews, and brief text messages are produced every day. Microblog data is typically of poor quality; hence improving the quality of the data is a significant scientific and practical challenge. In spite of the relevance of the problem, there has been not much work so far, especially in regard to microblog data quality for Short-Text Topic Modelling (STTM) purposes. This paper addresses this problem and proposes an approach called the social media data cleansing model (SMDCM) to improve data quality for STTM. We evaluate SMDCM using six topic modelling methods, namely the Latent Dirichlet Allocation (LDA), Word-Network Topic Model (WNTM), Pseudo-document-based Topic Modelling (PTM), Biterm Topic Model (BTM), Global and Local word embedding-based Topic Modeling (GLTM), and Fuzzy Topic modelling (FTM). We used the Real-world Cyberbullying Twitter (RW-CB-Twitter) and the Cyberbullying Mendeley (CB-MNDLY) datasets in the evaluation. The results proved the efficiency of the GLTM and WNTM over the other STTM models when applying the SMDCM techniques, which achieved optimum topic coherence and high accuracy values on RW-CB-Twitter and CB-MNDLY datasets.

show abstract

“…Where N denotes the number of documents in the dataset, K is the number of pre-defined hidden topics, v denotes to the number of available time-slices, and W represents the total number of words. Moreover, a new model called Noise BTM Word Embedding (NBTMWE) was developed by (Huang et al 2020) to tackle data sparsity. NBTMWE combines noise BTM and WE from external corpus to ameliorate the coherence of the topic.…”

Section: Global Word Co-occurrences Based Asttm Modelsmentioning

confidence: 99%

Short text topic modelling approaches in the context of big data: taxonomy, survey, and analysis

et al. 2022

View full text Add to dashboard Cite

Social media platforms such as (Twitter, Facebook, and Weibo) are being increasingly embraced by individuals, groups, and organizations as a valuable source of information. This social media generated information comes in the form of tweets or posts, and normally characterized as short text, huge, sparse, and low density. Since many real-world applications need semantic interpretation of such short texts, research in Short Text Topic Modeling (STTM) has recently gained a lot of interest to reveal unique and cohesive latent topics. This article examines the current state of the art in STTM algorithms. It presents a comprehensive survey and taxonomy of STTM algorithms for short text topic modelling. The article also includes a qualitative and quantitative study of the STTM algorithms, as well as analyses of the various strengths and drawbacks of STTM techniques. Moreover, a comparative analysis of the topic quality and performance of representative STTM models is presented. The performance evaluation is conducted on two real-world Twitter datasets: the Real-World Pandemic Twitter (RW-Pand-Twitter) dataset and Real-world Cyberbullying Twitter (RW-CB-Twitter) dataset in terms of several metrics such as topic coherence, purity, NMI, and accuracy. Finally, the open challenges and future research directions in this promising field are discussed to highlight the trends of research in STTM. The work presented in this paper is useful for researchers interested in learning state-of-the-art short text topic modelling and researchers focusing on developing new algorithms for short text topic modelling.

show abstract

Improving biterm topic model with word embeddings

Cited by 18 publications

References 32 publications

Identifying informative tweets during a pandemic via a topic-aware neural language model

Identifying informative tweets during a pandemic via a topic-aware neural language model

Enhancing Big Social Media Data Quality for Use in Short-Text Topic Modeling

Short text topic modelling approaches in the context of big data: taxonomy, survey, and analysis

Contact Info

Product

Resources

About