Target-specified Sequence Labeling with Multi-head Self-attention for Target-oriented Opinion Words Extraction

Abstract

Opinion target extraction and opinion term extraction are two fundamental tasks in Aspect Based Sentiment Analysis (ABSA). Many recent works on ABSA focus on Targetoriented Opinion Words (or Terms) Extraction (TOWE), which aims at extracting the corresponding opinion words for a given opinion target. TOWE can be further applied to Aspect-Opinion Pair Extraction (AOPE) which aims at extracting aspects (i.e., opinion targets) and opinion terms in pairs. In this paper, we propose Target-Specified sequence labeling with Multi-head Self-Attention (TSMSA) for TOWE, in which any pre-trained language model with multi-head self-attention can be integrated conveniently. As a case study, we also develop a Multi-Task structure named MT-TSMSA for AOPE by combining our TSMSA with an aspect and opinion term extraction module. Experimental results indicate that TSMSA outperforms the benchmark methods on TOWE significantly; meanwhile, the performance of MT-TSMSA is similar or even better than state-of-the-art AOPE baseline models.

Aspect-specific Context Modeling for Aspect-based Sentiment Analysis

“…It aims to extract from a sentence the corresponding opinion span describing an aspect. Most work in this area treats OE as a sequence tagging task, for which complex methods are developed to capture the interaction between the aspect and the context (Fan et al, 2019;Wu et al, 2020;Feng et al, 2021). More recent models such as TSMSA-BERT (Feng et al, 2021) and ARGCN-BERT (Jiang et al, 2021), adopt PLMs.…”

“…Most work in this area treats OE as a sequence tagging task, for which complex methods are developed to capture the interaction between the aspect and the context (Fan et al, 2019;Wu et al, 2020;Feng et al, 2021). More recent models such as TSMSA-BERT (Feng et al, 2021) and ARGCN-BERT (Jiang et al, 2021), adopt PLMs. In TSMSA-BERT, the multi-head self-attention is utilized to enhance the BERT PLM.…”

“…(d) ARGCN (Jiang et al, 2021) is an extension of R-GCNs suited to encode syntactic dependency information to complete OE. PLM-based models include: (a) BERT+Distance-rule (Feng et al, 2021) is the combination of BERT and Distance-rule. (b) TF-BERT (Feng et al, 2021) utilizes the average pooling of target word embeddings to represent the target information, then it is fed into BERT to extract target-oriented opinion terms.…”

“…PLM-based models include: (a) BERT+Distance-rule (Feng et al, 2021) is the combination of BERT and Distance-rule. (b) TF-BERT (Feng et al, 2021) utilizes the average pooling of target word embeddings to represent the target information, then it is fed into BERT to extract target-oriented opinion terms. (c) SDRN utilizes BERT as the encoder, which consists of an opinion entity extraction unit, a relation detection unit, and a synchronization unit for the aspect opinion pair extraction task.…”

“…(c) SDRN utilizes BERT as the encoder, which consists of an opinion entity extraction unit, a relation detection unit, and a synchronization unit for the aspect opinion pair extraction task. (d) TSMSA-BERT (Feng et al, 2021) uses a targetspecified sequence labeling method based on multihead self-attention (TSMSA) to perform OE. ARGCN+BERT (Jiang et al, 2021) adopts the last hidden states of the pretrained BERT as word representations and fine-tune it with the ARGCN model.…”