WING-NUS at SemEval-2017 Task 10: Keyphrase Extraction and Classification as Joint Sequence Labeling

Abstract

We describe an end-to-end pipeline processing approach for SemEval 2017's Task 10 to extract keyphrases and their relations from scientific publications. We jointly identify and classify keyphrases by modeling the subtasks as sequential labeling. Our system utilizes standard, surface-level features along with the adjacent word features, and performs conditional decoding on whole text to extract keyphrases.We focus only on the identification and typing of keyphrases (Subtasks A and B, together referred as extraction), but provide an end-to-end system inclusive of keyphrase relation identification (Subtask C) for completeness. Our top performing configuration achieves an F 1 of 0.27 for the end-to-end keyphrase extraction and relation identification scenario on the final test data, and compares on par to other top ranked systems for keyphrase extraction. Our system outperforms other techniques that do not employ global decoding and hence do not account for dependencies between keyphrases. We believe this is crucial for keyphrase classification in the given context of scientific document mining.

SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications

“…Teams Overall A B C s2 end2end (Ammar et al, 2017) 0.43 0.55 0.44 0.28 TIAL UW 0.42 0.56 0.44 TTI COIN (Tsujimura et al, 2017) 0.38 0.5 0.39 0.21 PKU ICL (Wang and Li, 2017) 0.37 0.51 0.38 0.19 NTNU-1 0.33 0.47 0.34 0.2 WING-NUS (Prasad and Kan, 2017) 0.27 0.46 0.33 0.04 Know-Center (Kern et al, 2017) 0.27 0.39 0.28 SZTE-NLP (Berend, 2017) 0.26 0.35 0.28 NTNU (Lee et al, 2017b) 0.23 0.3 0.24 0.08 LABDA (Flores et al, 2017) 0.04 0.08 0.04 upper bound 0.84 0.85 0.85 0.77 random 0.00 0.03 0.01 0.00 former is surprising, as keyphrases are with an overwhelming majority noun phrases, the latter not as much, many keyphrases only appear once in the dataset (see Table 1). GMBUAP further tried using empirical rules obtained by observing the training data for Subtask A, and a Naive Bayes classifier trained on provided training data for Subtask B.…”

Dataset Mention Extraction and Classification

“…The task asked to extract such entities and identify the relations among them on short excerpts of scientific documents. State-of-the-art deep learning and feature-based sequential labeling models set the standard for approaches on such tasks, using Long Short-Term Memory (LSTM) (Ammar et al, 2017) and Conditional Random Field (CRF) (Prasad and Kan, 2017) models, respectively.…”

SSN_MLRG1 at SemEval-2022 Task 10: Structured Sentiment Analysis using 2-layer BiLSTM

“…(Akhundov et al, 2018) have shown how BiLSTM-CRF can be used for this task. (Prasad and Kan, 2017) shows the extraction of keyphrases and relation prediction using CRFs for sequence labeling.…”