MRE: A Military Relation Extraction Model Based on BiGRU and Multi-Head Attention

Abstract

A great deal of operational information exists in the form of text. Therefore, extracting operational information from unstructured military text is of great significance for assisting command decision making and operations. Military relation extraction is one of the main tasks of military information extraction, which aims at identifying the relation between two named entities from unstructured military texts. However, the traditional methods of extracting military relations cannot easily resolve problems such as inadequate manual features and inaccurate Chinese word segmentation in military fields, failing to make full use of symmetrical entity relations in military texts. With our approach, based on the pre-trained language model, we present a Chinese military relation extraction method, which combines the bi-directional gate recurrent unit (BiGRU) and multi-head attention mechanism (MHATT). More specifically, the conceptual foundation of our method lies in constructing an embedding layer and combining word embedding with position embedding, based on the pre-trained language model; the output vectors of BiGRU neural networks are symmetrically spliced to learn the semantic features of context, and they fuse the multi-head attention mechanism to improve the ability of expressing semantic information. On the military text corpus that we have built, we conduct extensive experiments. We demonstrate the superiority of our method over the traditional non-attention model, attention model, and improved attention model, and the comprehensive evaluation value F1-score of the model is improved by about 4%.

Entity Linking Method for Chinese Short Texts with Multiple Embedded Representations

“…The term "Bi-directional" implies that the model processes information in both forward and backward directions. Specifically, BiGRU consists of two independent GRU layers: one processing the sequence in chronological order (forward GRU), and the other in reverse chronological order (backward GRU) [22]. The output at each time step is a concatenation of the outputs from the forward and backward GRUs, thereby incorporating both past and future contexts at each time step [23].…”

Models and Techniques for Domain Relation Extraction: A Survey

“…Then, a bidirectional gated recurrent unit network is used to capture the contextual information of the input sentences, and the attention mechanism is then used to focus on the most relevant parts of the input sentences for event detection. Lu et al (2021b) propose a military RE model using a combination of BiGRU and multi-head attention mechanisms. The method adopts a BiGRU-based NN to encode the input sentences and generate contextualized word representations.…”

Improving CNN-BGRU Hybrid Network for Arabic Handwritten Text Recognition

“…The weight is denoted by W , and the elementwise production is denoted by •. The BGRUs offer a mechanism to maximize the information processed [30,31]. It consists of concatenating the forward hidden layer and my backward hidden layer, which gives an output H t (formula (7)).…”