Bert-Based Text Keyword Extraction

Abstract

With the explosive growth of network information, in order to obtain the information faster and more accurately, this paper proposes a text keyword extraction method based on Bert. Firstly, the key sentence set is extracted from the background material by Bert model as the information supplement to the text. Then, based on the extended text, TF-IDF, text rank and LDA are combined to extract keywords. The experimental results on real science and technology academic paper data sets show that the performance of the fusion multi type feature combination algorithm is better than that of the traditional single algorithm; and the F value of the algorithm is increased by 1.5% by extracting key sentences from background materials, which further improves the effect of key word extraction.

RoBERTa-Based Keyword Extraction from Small Number of Korean Documents

“…Note that RoBERTa is encoder-centric since it is based on BERT. Thus, an additional keyword estimator is required to adopt RoBERTa in an end-toend style [22,23]. This paper uses a linear projection with softmax as the keyword estimator c(•) where it takes h i as an input and calculates the probability of h i 's being a keyword.…”

“…Supervised keyword extraction has been studied through two approaches: the classification-based approach and the generation-based approach. The classification-based approach involves extracting keywords from a document by evaluating every token in the document to determine if it is a keyword or not [21][22][23]. In contrast, the generation-based approach uses a generative language model to abstractively generate keywords for an input document [24,25].…”

A Spoken Dialogue Analysis Platform for Effective Counselling

“…Recently, Bidirectional Encoder Representations from Transformers (BERT) [27] has risen as a popular model for most natural language processing applications due to its high performance. In [28], the authors improved keyword analysis performance using BERT. BERT was used to extract key sentences from the given text as information that supplements the original text.…”

“…BERT was used to extract key sentences from the given text as information that supplements the original text. Then, statistical methods were applied to the extended text (the original text plus the key sentences) to extract keywords [28].…”