MDERank: A Masked Document Embedding Rank Approach for Unsupervised Keyphrase Extraction

Abstract

Keyphrase extraction (KPE) automatically extracts phrases in a document that provide a concise summary of the core content, which benefits downstream information retrieval and NLP tasks. Previous state-of-the-art (SOTA) methods select candidate keyphrases based on the similarity between learned representations of the candidates and the document. They suffer performance degradation on long documents due to discrepancy between sequence lengths which causes mismatch between representations of keyphrase candidates and the document. In this work, we propose a novel unsupervised embedding-based KPE approach, Masked Document Embedding Rank (MDERank), to address this problem by leveraging a mask strategy and ranking candidates by the similarity between embeddings of the source document and the masked document. We further develop a KPE-oriented BERT (KPEBERT) model by proposing a novel self-supervised contrastive learning method, which is more compatible to MDERank than vanilla BERT. Comprehensive evaluations on six KPE benchmarks demonstrate that the proposed MDERank outperforms state-of-the-art unsupervised KPE approach by average 1.80 F 1@15 improvement. MDERank further benefits from KPEBERT and overall achieves average 3.53 F 1@15 improvement over the SOTA SIFRank. Our code is available at https: //github.com/LinhanZ/mderank.

Improving Performance of Automatic Keyword Extraction (AKE) Methods Using PoS-Tagging and Enhanced Semantic-Awareness

“…While KP-Miner achieved a better performance for 14 of the 17 datasets with an average of 6.08% in F1-score, RaKUn was improved by a cross-dataset average of 4.46% for 14 of the 17 datasets. We observed the most change in (Sun et al, 2020) -SIFRank+ (Sun et al, 2020) -MDERank (Zhang et al, 2022) --…”

“…A more recent method is SIFRank (Sun et al, 2020), which combines sentence embedding model SIF and autoregressive pre-trained language model ELMo, and it was upgraded to SIFRank+ by position-biased weight to improve its performance for long documents. Lastly, MDERank (Zhang et al, 2022) considers the similarity between the embeddings of the source document and its masked version for candidate ranking.…”

Domain Knowledge-Based Analysis of Linux Vulnerability Characteristics and Evolution

“…First, we use the MDERank 46 algorithm to extract named entities. The MDERank algorithm selects keywords through candidate word masking and similarity calculation.…”

Present and absent keyphrases extraction: an approach based on sentence embedding

“…One of the recent methods that has achieved the best performance is the methods that use deep learning algorithms such as [23]- [25]. The development of sentence embedding techniques also contributed to the emergence of AKE methods that use these techniques as [26]- [28].…”

“…The keyphrases are selected from among the candidate keyphrases that have the greatest cosine similarity to the document using the maximal margin relevance, to avoid repetition of extracting the same keyphrases. MDERank [28], an unsupervised method that uses BERT technique [50] to embed the document and its variants. The principle of MDERank is to create variants for the original document while masking some phrases in these variants.…”