Neural Attention-Aware Hierarchical Topic Model

Jin, Yuan; Zhao, He; Liu, Ming; Du, Lan; Buntine, Wray L.

doi:10.48550/arxiv.2110.07161

Cited by 4 publications

(10 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, not all authors apply document classification or clustering algorithms before performing topic discovery. The results obtained in this paper are higher than the results obtained by the authors [8][9][10][12][13][14]. On the other hand, authors such as [14] obtain higher coherence values for the 20-Newsgroup corpus.…”

Section: Resultscontrasting

confidence: 84%

“…The results obtained in this paper are higher than the results obtained by the authors [8][9][10][12][13][14]. On the other hand, authors such as [14] obtain higher coherence values for the 20-Newsgroup corpus. The corpus Reuters [23,24] obtain higher coherence values than the results obtained in this work.…”

Section: Resultscontrasting

confidence: 84%

“…The corpus Reuters [23,24] obtain higher coherence values than the results obtained in this work. In [14]b and [24], the results are significant to the obtained in this paper because they apply algorithms and methods like variational automatic encoder and community detection and community mining. We consider the methods mentioned before to benefit the authors' results.…”

Section: Resultsmentioning

confidence: 67%

See 2 more Smart Citations

Integrating Text Classification in Topic Discovery with Semantic Embedding Models

Lezama-Sánchez¹,

Vidal²,

Reyes‐Ortiz³

2023

Preprint

View full text Add to dashboard Cite

Topic discovery is finding the main idea of large amounts of textual data. It indicates the recurring topics in the documents, allowing an overview of the texts. Current topic discovery models receive the texts, with or without pre-processing of Natural Language Processing. The processing consists of stopwords removal, text cleaning and normalization (lowercase conversion). A topic discovery model that receives texts with or without processing generates general topics since the input data is many uncategorized texts. The general topics do not offer a detailed overview of the input texts, and manual text categorization is a time-consuming and tedious task. Accordingly, it is necessary to integrate an automatic text classification task in the topic discovery process to obtain specific topics with their top words that contain relevant relationships based on belonging to a class. Text classification performs a word analysis that makes up a document to decide what class or category is being identified; then, integrating the text classification before a topic discovery process will provide latent topics depicted by top words with a high coherence in each topic based on the previously obtained classes. Therefore, this paper exposes a approach that integrates text classification into topic discovery from large amounts of English textual data, such as 20-Newsgroup and Reuters corpora. The text classification is accomplished with a Convolutional Neural Network(CNN) incorporating three embedding models based on semantic relationships. The topic discovery over categorized texts is realized with Latent Dirichlet Analysis(LDA), Probabilistic Latent Semantic Analysis(PLSA), and Latent Semantic Analysis(LSA) algorithms. An evaluation process was performed based on the normalized topic coherence metric. The 20-Newsgroup corpus was classified, and twenty topics with ten top words were discovered for each class, obtaining 0.1723 normalized topic coherence when applying LDA, 0.1622 with LSA, and 0.1716 with PLSA. The Reuters corpus was also classified, obtaining 0.1441 normalized topic coherence when applying the LDA algorithm to obtain 20 topics for each class.

show abstract

Section: Resultscontrasting

confidence: 84%

Section: Resultscontrasting

confidence: 84%

Section: Resultsmentioning

confidence: 67%

See 1 more Smart Citation

Integrating Text Classification in Topic Discovery with Semantic Embedding Models

Lezama-Sánchez¹,

Vidal²,

Reyes‐Ortiz³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Topic modeling has always been a catalyst for other research areas in Natural Language Process (NLP) (Panwar et al 2020;Jin et al 2021;Srivastava and Sutton 2016). A classic statistical topic model is Latent Dirichlet Allocation (LDA), which is based on Gibbs sampling to extract topics from documents (Blei, Ng, and Jordan 2003).…”

Section: Related Work Topic Modelmentioning

confidence: 99%

ConvNTM: Conversational Neural Topic Model

Sun

et al. 2023

AAAI

View full text Add to dashboard Cite

Topic models have been thoroughly investigated for multiple years due to their great potential in analyzing and understanding texts. Recently, researchers combine the study of topic models with deep learning techniques, known as Neural Topic Models (NTMs). However, existing NTMs are mainly tested based on general document modeling without considering different textual analysis scenarios. We assume that there are different characteristics to model topics in different textual analysis tasks. In this paper, we propose a Conversational Neural Topic Model (ConvNTM) designed in particular for the conversational scenario. Unlike the general document topic modeling, a conversation session lasts for multiple turns: each short-text utterance complies with a single topic distribution and these topic distributions are dependent across turns. Moreover, there are roles in conversations, a.k.a., speakers and addressees. Topic distributions are partially determined by such roles in conversations. We take these factors into account to model topics in conversations via the multi-turn and multi-role formulation. We also leverage the word co-occurrence relationship as a new training objective to further improve topic quality. Comprehensive experimental results based on the benchmark datasets demonstrate that our proposed ConvNTM achieves the best performance both in topic modeling and in typical downstream tasks within conversational research (i.e., dialogue act classification and dialogue response generation).

show abstract

“…Bahrainian S A et al [21] proposed a new light-weight Self-Supervised Neural Topic Model (SNTM) that learns a rich context by learning a topic representation jointly from three co-occurring words and a document that the triplet originates from. Jin Y et al [22] proposed a variational autoencoder (VAE) NTM model that jointly reconstructs the sentence and document word counts using combinations of bag-of-words (BoW) topical embeddings and pre-trained semantic embeddings. Zhao H et al [23] proposed to learn the topic distribution of a document by directly minimising its OT distance to the document's word distributions.…”

Section: Neural Topic Modelingmentioning

confidence: 99%

Neural Topic Modeling with Deep Mutual Information Estimation

Xu¹,

Lu²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

The emerging neural topic models make topic modeling more easily adaptable and extendable in unsupervised text mining. However, the existing neural topic models is difficult to retain representative information of the documents within the learnt topic representation. In this paper, we propose a neural topic model which incorporates deep mutual information estimation, i.e., Neural Topic Modeling with Deep Mutual Information Estimation(NTM-DMIE). NTM-DMIE is a neural network method for topic learning which maximizes the mutual information between the input documents and their latent topic representation. To learn robust topic representation, we incorporate the discriminator to discriminate negative examples and positive examples via adversarial learning. Moreover, we use both global and local mutual information to preserve the rich information of the input documents in the topic representation. We evaluate NTM-DMIE on several metrics, including accuracy of text clustering, with topic representation, topic uniqueness and topic coherence. Compared to the existing methods, the experimental results show that NTM-DMIE can outperform in all the metrics on the four datasets.

show abstract

Neural Attention-Aware Hierarchical Topic Model

Cited by 4 publications

References 22 publications

Integrating Text Classification in Topic Discovery with Semantic Embedding Models

Integrating Text Classification in Topic Discovery with Semantic Embedding Models

ConvNTM: Conversational Neural Topic Model

Neural Topic Modeling with Deep Mutual Information Estimation

Contact Info

Product

Resources

About