SentiWords: Deriving a High Precision and High Coverage Lexicon for Sentiment Analysis

Abstract

Deriving prior polarity lexica for sentiment analysis -where positive or negative scores are associated with words out of context -is a challenging task. Usually, a trade-off between precision and coverage is hard to find, and it depends on the methodology used to build the lexicon. Manually annotated lexica provide a high precision but lack in coverage, whereas automatic derivation from pre-existing knowledge guarantees high coverage at the cost of a lower precision. Since the automatic derivation of prior polarities is less time consuming than manual annotation, there has been a great bloom of these approaches, in particular based on the SentiWordNet resource. In this paper, we compare the most frequently used techniques based on SentiWordNet with newer ones and blend them in a learning framework (a so called 'ensemble method'). By taking advantage of manually built prior polarity lexica, our ensemble method is better able to predict the prior value of unseen words and to outperform all the other SentiWordNet approaches. Using this technique we have built SentiWords, a prior polarity lexicon of approximately 155,000 words, that has both a high precision and a high coverage. We finally show that in sentiment analysis tasks, using our lexicon allows us to outperform both the single metrics derived from SentiWordNet and popular manually annotated sentiment lexica.

Identification and handling of intensifiers for enhancing accuracy of Urdu sentiment analysis

“…In order to perform sentiment analysis, specialized lexica is used, which is mostly a list of positive and negative words for assigning scores to the input text (Gatti, Guerini, & Turchi, ). For performing Urdu sentiment analysis, a wide coverage Urdu lexicon is developed first.…”

“…In order to perform sentiment analysis, specialized lexica is used, which is mostly a list of positive and negative words for assigning scores to the input text (Gatti, Guerini, & Turchi, 2016 . For having maximum reliability, Urdu lughat is reused for validating the different POS assigned by the tagger.…”

Prior Polarity Dictionary Derived from SentiWordNet based on Random Forest Algorithm

“…The above 14 calculated feature values can be used as the sentiment strength score. At the same time, in [4], these features are input to support vector machine model to obtain the best prior polarity dictionary. The next section of our paper will apply Random Forest model to obtain a priori polarity dictionary which has superior performance than the dictionary obtained in [4].…”

“…Inspired by the paper [4], we put forward a novel method to obtain prior polarity dictionary with Random Forest algorithm. Combined with various sentiment calculation formulas the algorithm can get a sentiment dictionary with higher performance.…”

“…The experiments demonstrate that weighted average over word senses is the strategy that best approximates human judgment. Some authors follow the previous work and put forward the method based on support vector machine to obtain the sentiment strength of words [4,11]. The experiments demonstrate a better performance than the weighted average method in [10].…”