Bert-Based Latent Semantic Analysis (Bert-LSA): A Case Study on Geospatial Data Technology and Application Trend Analysis

Cheng, Quanying; Yunqiang, Zhu; Song, Jia; Zeng, Hongyun; Wang, Shu; Sun, Kai; Zhang, Jinqu

doi:10.3390/app112411897

Cited by 10 publications

(6 citation statements)

References 42 publications

(41 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [26], expose a new method of topic discovery. The method combines a pre-trained Bert model and a k-means spherical clustering algorithm and applies similarity between documents and topics.…”

Section: Related Workmentioning

confidence: 99%

Integrating Text Classification in Topic Discovery with Semantic Embedding Models

Lezama-Sánchez¹,

Vidal²,

Reyes‐Ortiz³

2023

Preprint

View full text Add to dashboard Cite

Topic discovery is finding the main idea of large amounts of textual data. It indicates the recurring topics in the documents, allowing an overview of the texts. Current topic discovery models receive the texts, with or without pre-processing of Natural Language Processing. The processing consists of stopwords removal, text cleaning and normalization (lowercase conversion). A topic discovery model that receives texts with or without processing generates general topics since the input data is many uncategorized texts. The general topics do not offer a detailed overview of the input texts, and manual text categorization is a time-consuming and tedious task. Accordingly, it is necessary to integrate an automatic text classification task in the topic discovery process to obtain specific topics with their top words that contain relevant relationships based on belonging to a class. Text classification performs a word analysis that makes up a document to decide what class or category is being identified; then, integrating the text classification before a topic discovery process will provide latent topics depicted by top words with a high coherence in each topic based on the previously obtained classes. Therefore, this paper exposes a approach that integrates text classification into topic discovery from large amounts of English textual data, such as 20-Newsgroup and Reuters corpora. The text classification is accomplished with a Convolutional Neural Network(CNN) incorporating three embedding models based on semantic relationships. The topic discovery over categorized texts is realized with Latent Dirichlet Analysis(LDA), Probabilistic Latent Semantic Analysis(PLSA), and Latent Semantic Analysis(LSA) algorithms. An evaluation process was performed based on the normalized topic coherence metric. The 20-Newsgroup corpus was classified, and twenty topics with ten top words were discovered for each class, obtaining 0.1723 normalized topic coherence when applying LDA, 0.1622 with LSA, and 0.1716 with PLSA. The Reuters corpus was also classified, obtaining 0.1441 normalized topic coherence when applying the LDA algorithm to obtain 20 topics for each class.

show abstract

Section: Related Workmentioning

confidence: 99%

Integrating Text Classification in Topic Discovery with Semantic Embedding Models

Lezama-Sánchez¹,

Vidal²,

Reyes‐Ortiz³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The authors of [19,[25][26][27][28][29][30] applied word embedding to discover topics in long and short texts, or they applied clustering algorithms. Hence, in [25,26], the authors showed models based on word embeddings.…”

Section: Related Workmentioning

confidence: 99%

Integrating Text Classification into Topic Discovery Using Semantic Embedding Models

Lezama-Sánchez,

Tovar Vidal,

Reyes-Ortiz

2023

Applied Sciences

View full text Add to dashboard Cite

Topic discovery involves identifying the main ideas within large volumes of textual data. It indicates recurring topics in documents, providing an overview of the text. Current topic discovery models receive the text, with or without pre-processing, including stop word removal, text cleaning, and normalization (lowercase conversion). A topic discovery process that receives general domain text with or without processing generates general topics. General topics do not offer detailed overviews of the input text, and manual text categorization is tedious and time-consuming. Extracting topics from text with an automatic classification task is necessary to generate specific topics enriched with top words that maintain semantic relationships among them. Therefore, this paper presents an approach that integrates text classification for topic discovery from large amounts of English textual data, such as 20-Newsgroups and Reuters Corpora. We rely on integrating automatic text classification before the topic discovery process to obtain specific topics for each class with relevant semantic relationships between top words. Text classification performs a word analysis that makes up a document to decide what class or category to identify; then, the proposed integration provides latent and specific topics depicted by top words with high coherence from each obtained class. Text classification accomplishes this with a convolutional neural network (CNN), incorporating an embedding model based on semantic relationships. Topic discovery over categorized text is realized with latent Dirichlet analysis (LDA), probabilistic latent semantic analysis (PLSA), and latent semantic analysis (LSA) algorithms. An evaluation process for topic discovery over categorized text was performed based on the normalized topic coherence metric. The 20-Newsgroups corpus was classified, and twenty topics with the ten top words were identified for each class. The normalized topic coherence obtained was 0.1723 with LDA, 0.1622 with LSA, and 0.1716 with PLSA. The Reuters Corpus was also classified, and twenty and fifty topics were identified. A normalized topic coherence of 0.1441 was achieved when applying the LDA algorithm, obtaining 20 topics for each class; with LSA, the coherence was 0.1360, and with PLSA, it was 0.1436.

show abstract

“…Ref. [7] applies BERT word embeddings and a classical clustering algorithm (spherical k-means) to assign documents to topics. The proposal encodes documents as a linear combination of word embeddings and word frequencies in the document.…”

Section: Text Classificationmentioning

confidence: 99%

Current Approaches and Applications in Natural Language Processing

Montejo-Ráez

Jiménez-Zafra

2022

Applied Sciences

View full text Add to dashboard Cite

show abstract

Bert-Based Latent Semantic Analysis (Bert-LSA): A Case Study on Geospatial Data Technology and Application Trend Analysis

Cited by 10 publications

References 42 publications

Integrating Text Classification in Topic Discovery with Semantic Embedding Models

Integrating Text Classification in Topic Discovery with Semantic Embedding Models

Integrating Text Classification into Topic Discovery Using Semantic Embedding Models

Current Approaches and Applications in Natural Language Processing

Contact Info

Product

Resources

About