Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis

Tang, Jialong; Lu, Zhiyong; Su, Jinsong; Ge, Yubin; Song, Linfeng; Sun, Le; Luo, Jiebo

doi:10.18653/v1/p19-1053

Cited by 85 publications

(55 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MN(+ AS) [45]: MN(+AS) is designed based on MN, the context attention weight extracts the correct / wrong prediction for each instance at each iteration, and then mask this word to continue the iteration.…”

Section: Experimental Comparison Modelsmentioning

confidence: 99%

“…TNET-ATT(+AS) [45]: TNET-ATT(+AS) is designed based on TNET, it uses the same technology as MN to deal with the weight of context attention. The results of our model and baseline models on datasets Restaurant, Laptop and Twitter are shown in Table 2, we find that IRAN has better performance than most frontier models on Accuracy and F1-Measure, which verifies the effectiveness of our model.…”

Section: Experimental Comparison Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Interactive Rule Attention Network for Aspect-Level Sentiment Analysis

Zhu

Zhang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Aspect-level sentiment analysis is a fundamental task in NLP, and it aims to predict the sentiment polarity of each specific aspect term in a given sentence. Recent researches show that the finegrained sentiment analysis for aspect-level has become a research hotspot. However, previous work did not consider the influence of grammatical rules on aspect-level sentiment analysis. In addition, attention mechanism is too simple to learn attention information from context and target interactively. Therefore, we propose an interactive rule attention network (IRAN) for aspect-level sentiment analysis. IRAN not only designs a grammar rule encoder, which simulates the grammatical functions at the sentence by standardizing the output of adjacent positions, but also constructs an interaction attention network to learn attention information from context and target. Experimental results on SemEval 2014 Dataset and ACL 2014 Twitter Dataset demonstrate IRAN can learn effective features and obtain superior performance over the baseline models. INDEX TERMS Aspect-level sentiment analysis, grammatical rules, IRAN, interaction attention network.

show abstract

Section: Experimental Comparison Modelsmentioning

confidence: 99%

Section: Experimental Comparison Modelsmentioning

confidence: 99%

Interactive Rule Attention Network for Aspect-Level Sentiment Analysis

Zhu

Zhang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Gu et al [19] proposed a Position-Aware Bidirectional Attention Network (PBAN), which not only concentrates on the position information of aspect terms but also mutually models the relation between aspect terms and sentence by employing bidirectional attention mechanism. Tang et al [20] addressed the problems of strong mode over-learning and weak mode under-learning in the neural network learning process and proposed a progressive self-supervised attention mechanism algorithm. They properly constrained the attention mechanism.…”

Section: Related Workmentioning

confidence: 99%

Multiple Interactive Attention Networks for Aspect-Based Sentiment Classification

Zhang

Zhu

et al. 2020

Applied Sciences

View full text Add to dashboard Cite

Aspect-Based (also known as aspect-level) Sentiment Classification (ABSC) aims at determining the sentimental tendency of a particular target in a sentence. With the successful application of the attention network in multiple fields, attention-based ABSC has aroused great interest. However, most of the previous methods are difficult to parallelize, insufficiently obtain, and fuse the interactive information. In this paper, we proposed a Multiple Interactive Attention Network (MIN). First, we used the Bidirectional Encoder Representations from Transformers (BERT) model to pre-process the data. Then, we used the partial transformer to obtain a hidden state in parallel. Finally, we took the target word and the context word as the core to obtain and fuse the interactive information. Experimental results on the different datasets showed that our model was much more effective.Appl. Sci. 2020, 10, 2052 2 of 15 words [7]. However, these models simply average the aspect or context vector to guide learning the attention weight on the context or aspect words. Therefore, these models are still in the preliminary stage in dealing with fine-grained sentiment analysis.In conclusion, there are two problems with previous approaches. The first problem is that previous approaches are difficult to obtain the hidden state interactively in parallel. Another problem is to insufficiently obtain and fuse contextual information and aspect information.This paper proposed a model named Multiple Interactive Attention Network (MIN) to address these problems. To address the first problem, we took advantage of Multi-Head Attention (MHA) to obtain useful interactive information. To address another problem, we adopted target-context pair and Context-Target-Interaction (CTI) in our model.The main contributions of this paper are presented as follows:1.We took advantage of MHA and Location-Point-Wise Feed-Forward Networks (LPFFN) to obtain the hidden state interactively in parallel. Besides, we applied pre-trained Bidirectional Encoder Representations from Transformers (BERT) [8] in our model. 2.We used the CTI and target-context pair to help us obtain and fuse useful information. We also verified the effectiveness of these two methods. 3.We experimented on different public authoritative datasets: restaurant reviews and laptop reviews of the SemEval-2014 Task 4 dataset, the ACL(Annual Meeting of the Association for Computational Linguistics) 14 Twitter dataset, SemEval-2015 Task 12 dataset, SemEval-2016 Task 5 dataset. The experimental results showed our model outperformed state-of-the-art methods.

show abstract

“…There are kinds of sentiment analysis tasks, such as documentlevel (Thongtan and Phienthrakul, 2019), sentence-level 4 , aspect-level (Pontiki et al, 2014;Wang et al, 2019a) and multimodal (Chen et al, 2018;Akhtar et al, 2019) sentiment analysis. For the aspect-level sentiment analysis, previous work typically apply attention mechanism (Luong et al, 2015) combining with memory network (Weston et al, 2014) or gating units to solve this task (Tang et al, 2016b;He et al, 2018a;Xue and Li, 2018;Duan et al, 2018;Tang et al, 2019;Yang et al, 2019;Bao et al, 2019), where an aspect-independent encoder is used to generate the sentence representation. In addition, some work leverage the aspect-weakly associative encoder to generate aspect-specific sentence representation (Tang et al, 2016a;Wang et al, 2016;Majumder et al, 2018).…”

Section: Related Workmentioning

confidence: 99%

A Novel Aspect-Guided Deep Transition Model for Aspect Based Sentiment Analysis

Liang¹,

Meng²,

Zhang³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Aspect based sentiment analysis (ABSA) aims to identify the sentiment polarity towards the given aspect in a sentence, while previous models typically exploit an aspectindependent (weakly associative) encoder for sentence representation generation. In this paper, we propose a novel Aspect-Guided Deep Transition model, named AGDT, which utilizes the given aspect to guide the sentence encoding from scratch with the speciallydesigned deep transition architecture. Furthermore, an aspect-oriented objective is designed to enforce AGDT to reconstruct the given aspect with the generated sentence representation. In doing so, our AGDT can accurately generate aspect-specific sentence representation, and thus conduct more accurate sentiment predictions. Experimental results on multiple SemEval datasets demonstrate the effectiveness of our proposed approach, which significantly outperforms the best reported results with the same setting 1 .

show abstract

Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis

Cited by 85 publications

References 21 publications

Interactive Rule Attention Network for Aspect-Level Sentiment Analysis

Interactive Rule Attention Network for Aspect-Level Sentiment Analysis

Multiple Interactive Attention Networks for Aspect-Based Sentiment Classification

A Novel Aspect-Guided Deep Transition Model for Aspect Based Sentiment Analysis

Contact Info

Product

Resources

About