Aspect Level Sentiment Classification with Deep Memory Network

Abstract

We introduce a deep memory network for aspect level sentiment classification. Unlike feature-based SVM and sequential neural models such as LSTM, this approach explicitly captures the importance of each context word when inferring the sentiment polarity of an aspect. Such importance degree and text representation are calculated with multiple computational layers, each of which is a neural attention model over an external memory. Experiments on laptop and restaurant datasets demonstrate that our approach performs comparable to state-of-art feature based SVM system, and substantially better than LSTM and attention-based LSTM architectures. On both datasets we show that multiple computational layers could improve the performance. Moreover, our approach is also fast. The deep memory network with 9 layers is 15 times faster than LSTM with a CPU implementation.

Recurrent Attention Network on Memory for Aspect Sentiment Analysis

“…• MemNet (Tang et al, 2016): It applies attention multiple times on the word embeddings, and the last attention's output is fed to softmax for prediction, without combining the results of different attentions. We produce its results on all four datasets with the code released by the authors.…”

“…Single attention or multiple attentions were applied in aspect sentiment classification in some previous works (Wang et al, 2016;Tang et al, 2016). One difference between our method and (Tang et al, 2016) is that we introduce a memory module between the attention module and the input module, thus our method can synthesize features of word sequences such as sentiment phrases (e.g. "not wonderful enough").…”

“…MemNet (Tang et al, 2016) simply used the sequence of word vectors as memory, which cannot synthesize phrase-like features in the original sentence. It is straightforward to achieve the goal with the models of RNN family.…”

“…We utilize the previous episode for the current attention, since it can guide the model to attend different useful information. (Tang et al, 2016) also adopts multiple attentions, but they don't combine the results of different attentions. Then we calculate the normalized attention score of each memory slice as:…”

“…For the first two datasets, we removed a few examples having the "conflict label", e.g., "Certainly not the best sushi in New York, however, it is always fresh" (Pontiki et al, 2014). We use 300-dimension word vectors pre-trained by GloVe (Pennington et al, 2014) (whose vocabulary size is 1.9M 2 ) for our experiments on the English datasets, as previous works did (Tang et al, 2016). Some works employed domain-specific training corpus to learn embeddings for better performance, such as TD-LSTM (Tang et al, 2015) on the tweet dataset.…”