A new approach to query segmentation for relevance ranking in web search

Abstract

In this paper, we try to determine how best to improve state-of-the-art methods for relevance ranking in web searching by query segmentation. Query segmentation is meant to separate the input query into segments, typically natural language phrases. We propose employing the re-ranking approach in query segmentation, which first employs a generative model to create the top k candidates and then employs a discriminative model to re-rank the candidates to obtain the final segmentation result. The method has been widely utilized for structure prediction in natural language processing, but has not been applied to query segmentation, as far as we know. Furthermore, we propose a new method for using the results of query segmentation in relevance ranking, which takes both the original query words and the segmented query phrases as units of query representation. We investigate whether our method can improve three relevance models, namely n-gram BM25, key n-gram model and term dependency model, within the framework of learning to rank. Our experimental results on large scale web search datasets show that our method can indeed significantly improve relevance ranking in all three cases.

Senti2vec: An Effective Feature Extraction Technique for Sentiment Analysis Based on Word2vec

“…In contrast, Fan et al used Naive Bayes as the classification method to build sentiment lexicons through word vectors matrices separately and then used the Boolean rules to classify the matched documents for polarity that appeared in both matrices [38]. An extended model for sentiment classification [16], is presented by Haocheng et al in [42], which focused on the semantic features between words rather than the simple lexical or syntactic features. For micro-blog, Zhang et al investigated the use of multi-label classification, two micro-blog datasets, and eight different evaluation matrices on three different sentiment dictionaries [43].…”