Extraction of Joint Entity and Relationships with Soft Pruning and GlobalPointer

Abstract

In recent years, scholars have paid increasing attention to the joint entity and relation extraction. However, the most difficult aspect of joint extraction is extracting overlapping triples. To address this problem, we propose a joint extraction model based on Soft Pruning and GlobalPointer, short for SGNet. In the first place, the BERT pretraining model is used to obtain the text word vector representation with contextual information, and then the local and non-local information of the word vector is obtained through graph operations. Specifically, to address the lack of information caused by the rule-based pruning strategies, we utilize the Gaussian Graph Generator and the attention-guiding layer to construct a fully connected graph. This process is called soft pruning for short. Then, to achieve node message passing and information integration, we employ GCNs and a thick connection layer. Next, we use the GlobalPointer decoder to convert triple extraction into quintuple extraction to tackle the problem of problematic overlapping triples extraction. The GlobalPointer decoder, unlike the typical feedforward neural network (FNN), can perform joint decoding. In the end, to evaluate the model performance, the experiment was carried out on two public datasets: the NYT and WebNLG. The experiments show that SGNet performs substantially better on overlapping extraction and achieves good results on two publicly available datasets.

Multi-Model Fusion-Based Hierarchical Extraction for Chinese Epidemic Event

“…BERT-GlobalPointer In pointer network designed for named entity recognition, we usually utilize two modules to identify the head and tail of the entity respectively, which leads to inconsistent training and prediction. GlobalPointer [ 21 ] treats both ends as a whole to deal with such inconsistencies. Therefore, GlobalPointer has a more global view.…”

“…There are two mainstreams of deep learning-based EE approaches: (1) Pipeline-based approach [3,10,21,22,24,28,39,45] that first performs ETI and then identifies arguments base on the results of ETI. (2) Joint-based approach [16,25,29,31,44,47,49] that treats EE as a structure extraction task, and predicts event type and corresponding arguments at the same time.…”

Joint Extraction of Entities and Relations via Entity and Relation Heterogeneous Graph Attention Networks

“…A heterogeneous graph attention network is constructed including three types of nodes-word nodes, relation nodes, and subject nodes-to learn and enhance the embedded representation of parts for relation extraction, combined with a gating mechanism to control the updating of the nodes. The experimental results demonstrate that the ERHGA can effectively solve the relational triple extraction problem and outperform all baselines [1][2][3][8][9][10][11][12][13][14][15].…”

“…RMAN [14] applied two multi-head attention layers to add extra semantic information. A Gaussian graph generator and an attention-guiding layer were utilized to address the lack of information, and the GlobalPointer decoder was used to tackle the extraction of problematic overlapping triples by Liang et al [15].…”

“…To ensure the fairness of experimental comparison, we used the most commonly used benchmark dataset WebNLG [36], as used in [1][2][3][8][9][10][11][12][13][14][15]37]. The WebNLG dataset was obtained via natural language processing.…”