Markov Blankets and Meta-heuristics Search: Sentiment Extraction from Unstructured Texts

Abstract

Abstract. Extracting sentiments from unstructured text has emerged as an important problem in many disciplines. An accurate method would enable us, for example, to mine online opinions from the Internet and learn customers' preferences for economic or marketing research, or for leveraging a strategic advantage. In this paper, we propose a two-stage Bayesian algorithm that is able to capture the dependencies among words, and, at the same time, finds a vocabulary that is efficient for the purpose of extracting sentiments. Experimental results on online movie reviews and online news show that our algorithm is able to select a parsimonious feature set with substantially fewer predictor variables than in the full data set and leads to better predictions about sentiment orientations than several state-of-the-art machine learning methods. Our findings suggest that sentiments are captured by conditional dependence relations among words, rather than by keywords or high-frequency words.

Opinion Mining and Sentiment Analysis

“…Riloff et al [254] explore the use of a subsumption hierarchy to formally define different types of lexical features and the relationships between them in order to identify useful complex features for opinion analysis. Airoldi et al [5] apply a Markov Blanket Classifier to this problem together with a meta-heuristic search strategy called Tabu search to arrive at a dependency structure encoding a parsimonious vocabulary for the positive and negative polarity classes.…”

“…For example, Pang et al [235] compare Naive Bayes, Support Vector Machines, and maximum-entropy-based classification on the sentiment-polarity classification problem for movie reviews. More extensive comparisons of the performance of standard machine learning techniques with other types of features or feature selection schemes have been engaged in later work [5,69,103,204,217]; see Section 4.2 for more detail. We note that there has been some research that explicitly considers regression or ordinal-regression formulations of opinion-mining problems [109,201,233,320]: example questions include, "how positive is this text?"…”

“…In early work, Hatzivassiloglou and McKeown [119] present an approach based on linguistic heuristics. 5 Their technique is built on the fact that in the case of polarity classification, the two classes of interest represent opposites, and we can utilize "opposition constraints" to help make labeling decisions. Specifically, constraints between pairs of adjectives are induced from a large corpus by looking at whether the two words are linked by conjunctions such as "but" (evidence for opposing orientations: "elegant but over-priced") or "and" (evidence for the same orientation: "clever and informative").…”

“…TV, which allow the user to sort reviews by helpfulness; CitySearch, which solicits utility evaluations from general users and gives more helpful reviews greater prominence; and Epinions, which only allows registered members to rate reviews and does not appear to have helpfulness as a sort criterion, at least for nonregistered visitors. 5 (We learned about the solicitation of utility evaluations by IMDb from Zhuang et al [332] and by Citysearch from Dellarocas [71]. )…”

Reprint of: Computational approaches for mining user’s opinions on the Web 2.0

“…Choi & Cardie, 2010;Li et al, 2010), latent semantic association (e.g. Guo, Zhu, Guo, & Su, 2011;Guo, Zhu, Guo, Zahng, & Su, 2009;Hofmann, 2001) up to combinations and variations of existing algorithms (Airoldi, Bai, & Padman, 2006;Choi, Cardie, Riloff, & Patwardhan, 2005;Jakob & Gurevych, 2010;Nakagawa, Inui, & Kurohashi, 2010). One important part of the preprocessing steps in opinion mining is Part-of-Speech (POS) tagging; a variety of algorithms can be applied to implement this task: rule based approaches, Markov model approaches, maximum entropy approaches, etc.…”