Bayes-enhanced Lifelong Attention Networks for Sentiment Classification

Abstract

The classic deep learning paradigm learns a model from the training data of a single task and the learned model is also tested on the same task. This paper studies the problem of learning a sequence of tasks (sentiment classification tasks in our case). After each sentiment classification task is learned, its knowledge is retained to help future task learning. Following this setting, we explore attention neural networks and propose a Bayes-enhanced Lifelong Attention Network (BLAN). The key idea is to exploit the generative parameters of naïve Bayes to learn attention knowledge. The learned knowledge from each task is stored in a knowledge base and later used to build lifelong attentions. The constructed lifelong attentions are then used to enhance the attention of the network to help new task learning. Experimental results on product reviews from Amazon.com show the effectiveness of the proposed model.

A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges

“…Wang et al [191] impose the idea of lifelong learning into the ASC task and propose a novel lifelong learning approach based on memory networks. Recent lifelong sentiment analysis studies begin to investigate the catastrophic forgetting issue during the sequential learning [192,193,194,195], instead of simply studying it as an extension of crossdomain sentiment analysis for knowledge accumulation. However, existing studies mainly focus on domain incremental learning for the ASC task [194,195], where all tasks sharing the same fixed label classes (e.g., positive, negative, and neutral) and no task information is required.…”

Enhancing sentence semantic matching with isotropic batch normalization and generalized pooling operator

“…Additionally, they used a word grid graph as input to retain multigranularity information, serving as a complement to pre-trained language models. Wang et al introduced a novel model called Dabert, which improves sentence semantic modeling through distance-aware self-attention and multi-level matching [6]. The model considers the importance of word distances and captures interactions between sentences.…”