Twitter Sentiment Analysis with Deep Convolutional Neural Networks

Abstract

This paper describes our deep learning system for sentiment analysis of tweets. The main contribution of this work is a new model for initializing the parameter weights of the convolutional neural network, which is crucial to train an accurate model while avoiding the need to inject any additional features. Briefly, we use an unsupervised neural language model to train initial word embeddings that are further tuned by our deep learning model on a distant supervised corpus. At a final stage, the pre-trained parameters of the network are used to initialize the model. We train the latter on the supervised training data recently made available by the official system evaluation campaign on Twitter Sentiment Analysis organized by Semeval-2015. A comparison between the results of our approach and the systems participating in the challenge on the official test sets, suggests that our model could be ranked in the first two positions in both the phrase-level subtask A (among 11 teams) and on the message-level subtask B (among 40 teams). This is an important evidence on the practical value of our solution.

Cross-Lingual Classification of Topics in Political Texts

“…Recently, convolutional neural networks (LeCun and Bengio, 1998, CNN) have yielded best performance on many text classification tasks (Kim, 2014;Severyn and Moschitti, 2015). CNN is a feed-forward neural network consisting of one or more convolution layers.…”

SwissCheese at SemEval-2016 Task 4: Sentiment Classification Using an Ensemble of Convolutional Neural Networks with Distant Supervision

“…Deep neural networks have shown great promises at capturing salient features for these complex tasks (Mikolov et al, 2013b;Severyn and Moschitti, 2015a). Particularly successful for sentiment classification were Convolutional Neural Networks (CNN) (Kim, 2014;Kalchbrenner et al, 2014;Severyn and Moschitti, 2015a;Severyn and Moschitti, 2015b;Johnson and Zhang, 2015), on which our work builds upon. * These authors contributed equally to this work These networks typically have a large number of parameters and are especially effective when trained on large amounts of data.…”

“…* These authors contributed equally to this work These networks typically have a large number of parameters and are especially effective when trained on large amounts of data. In this work, we use a distant supervision approach to leverage large amounts of data in order to train a 2-layer CNN 1 , extending the 1-layer architecture proposed by (Severyn and Moschitti, 2015a). More specifically, we train a neural network using the following three-phase procedure: i) creation of word embeddings for initialization of the first layer; ii) distant supervised phase, where the network weights and word embeddings are trained to capture aspects related to sentiment; and iii) supervised phase, where the network is trained on the provided supervised training data.…”

Sentiment Detection, Recognition and Aspect Identification

“…While Reverse-JST always performs worse than JST meaning that the JST model is more suitable for sentiment topic detection. The sentiment detection based on word embeddings and the use of a Deep Convolutional Neural Network developed with the help of an unsupervised neural language model, such as in Severyn and Moschitti [14].…”