Research on the Evolution of Journal Topic Mining Based on the BERT-LDA Model

Abstract

Scientific papers are an important form for researchers to summarize and display their research results. Information mining and analysis of scientific papers can help to form a comprehensive understanding of the subject. Aiming at the ignorance of contextual semantic information in current topic mining and the uncertainty of screening rules in association evolution research, this paper proposes a topic mining evolution model based on the BERT-LDA model. First, the model combines the contextual semantic information learned by the BERT model with the word vectors of the LDA model to mine deep semantic topics. Then construct topic filtering rules to eliminate invalid associations between topics. Finally, the relationship between themes is analyzed through the theme evolution, and the complex relationship between the themes such as fusion, diffusion, emergence, and disappearance is displayed. The experimental results show that, compared with the traditional LDA model, the topic mining evolution model based on BERTLDA can accurately mine topics with deep semantics and effectively analyze the development trend of scientific and technological paper topics.

Advancements of Artificial Intelligence Techniques in the Realm About Library and Information Subject—A Case Survey of Latent Dirichlet Allocation Method

“…Consequently, it has been widely applied to the discovery of topics within policy texts, spanning domains such as climate [71] and government open data [72]. Scholars have further leveraged the LDA2vec model to enhance the comprehensiveness of semantic content extraction within policy texts [73]. Furthermore, labels, the terms used to classify or describe web information resources, have been successfully generated using the LDA method in contexts like Weibo [74] and online healthcare [75].…”

Topic Modeling for Scientific Articles: Exploring Optimal Hyperparameter Tuning in BERT

“…A prominent example is Bidirectional Encoder Representations from Transformers (BERT) is designed to pretrain deep bidirectional representations from unlabeled text. BERT's ability to capture contextual semantic significantly improves the depth and accuracy of topic mining, overcoming limitations of traditional LDA which might ignore such context [18]. Through its attention mechanisms, BERT can automatically form topical word cluster similar to those generated by LDA [19].…”