Relation Extraction Based on Prompt Information and Feature Reuse

Abstract

To alleviate the problem of under-utilization features of sentence-level relation extraction, which leads to insufficient performance of the pre-trained language model and underutilization of the feature vector, a sentence-level relation extraction method based on adding prompt information and feature reuse is proposed. At first, in addition to the pair of nominals and sentence information, a piece of prompt information is added, and the overall feature information consists of sentence information, entity pair information, and prompt information, and then the features are encoded by the pre-trained language model ROBERTA. Moreover, in the pre-trained language model, BIGRU is also introduced in the composition of the neural network to extract information, and the feature information is passed through the neural network to form several sets of feature vectors. After that, these feature vectors are reused in different combinations to form multiple outputs, and the outputs are aggregated using ensemble-learning soft voting to perform relation classification. In addition to this, the sum of cross-entropy, KL divergence, and negative log-likelihood loss is used as the final loss function in this paper. In the comparison experiments, the model based on adding prompt information and feature reuse achieved higher results of the SemEval-2010 task 8 relational dataset.

Joint Entity and Relation Extraction Model Based on Inner and Outer Tensor Dot Product and Single-Table Filling

“…Feature Extraction Layer. We use the representation H from BERT's final layer output as input to the BiGRU [17] model, which enables capturing more semantic information, enhancing token-level representations, and obtaining the vector representation Y:…”

“…This is because focal loss assigns more weight to hard samples, making the model pay more attention to them, which is very helpful for this table-filling method. In the fourth group, we replaced the BiGRU [17] model with the BiLSTM model and found that compared to the BiLSTM model, BiGRU worked better in combination with the attention mechanism, resulting in better performance. Figure 4 shows shows the convergence process of various ablation models on the NYT dataset.…”

“…This means that we construct continuous subject-relation pair labels and object-relation pair labels to identify the spans of subject and object entities, thereby addressing the problem of complex triple overlap. To capture more global information, BiGRU [17] is employed to extract additional hidden layer information, and a multi-head tensor dot-product operation is employed to encode table features. These encoded features are subsequently fused with the output of the self-attention mechanism within the transformer [18] to iteratively acquire the intermediate encoding of table cells.…”

Leveraging Prompt and Top-K Predictions with ChatGPT Data Augmentation for Improved Relation Extraction

“…Ref. [14] proposes a relational extraction method that adds prompt information and feature reuse. Firstly, the prompt information is added before each sentence.…”