Multi-Grained Attention Representation With ALBERT for Aspect-Level Sentiment Classification

Abstract

Aspect-level sentiment classification aims to solve the problem, which is to judge the sentiment tendency of each aspect in a sentence with multiple aspects. Previous works mainly employed Long Short-Term Memory (LSTM) and Attention mechanisms to fuse information between aspects and sentences, or to improve large language models such as BERT to adapt aspect-level sentiment classification tasks. The former methods either did not integrate the interactive information of related aspects and sentences, or ignored the feature extraction of sentences. This paper proposes a novel multi-grained attention representation with ALBERT (MGAR-ALBERT). It can learn the representation that contains the relevant information of the sentence and the aspect, while integrating it into the process of sentence modeling with multi granularity, and finally get a comprehensive sentence representation. In Masked LM (MLM) task, in order to avoid the influence of aspect words being masked in the initial stage of the pre-training, the noise linear cosine decay is introduced into n − gram. We implemented a series of comparative experiments to verify the effectiveness of the method. The experimental results show that our model can achieve excellent results on Restaurant dataset with numerous number of parameters reduced, and it is not inferior to other models on Laptop dataset.INDEX TERMS Aspect-level sentiment classification, ALBERT, natural language processing, deep learning.

Aspect based sentiment analysis using fine-tuned BERT model with deep context features

“…The proposed model is evaluated on the system with Windows 10 platform packed with 8GB RAM and 2GB of compute unified device architecture (CUDA) enables NVIDIA graphics. To evaluate the model, accuracy and macro-F1 score is considered as evaluation parameter, also comparative analysis is carried out with the existing BERT model [20] to prove the model's efficiency.…”