Learning query intent from regularized click graphs

Abstract

This work presents the use of click graphs in improving query intent classifiers, which are critical if vertical search and general-purpose search services are to be offered in a unified user interface. Previous works on query classification have primarily focused on improving feature representation of queries, e.g., by augmenting queries with search engine results. In this work, we investigate a completely orthogonal approach -instead of enriching feature representation, we aim at drastically increasing the amounts of training data by semi-supervised learning with click graphs. Specifically, we infer class memberships of unlabeled queries from those of labeled ones according to their proximities in a click graph. Moreover, we regularize the learning with click graphs by content-based classification to avoid propagating erroneous labels. We demonstrate the effectiveness of our algorithms in two different applications, product intent and job intent classification. In both cases, we expand the training data with automatically labeled queries by over two orders of magnitude, leading to significant improvements in classification performance. An additional finding is that with a large amount of training data obtained in this fashion, classifiers using only query words/phrases as features can work remarkably well.

Task complexity, vertical display and user interaction in aggregated search

“…Existing methods for vertical selection and presentation use machine learning to combine different types of predictive evidence: query-string features [2,4,5,19,23], vertical query-log features [2,4,5,11,23], vertical content features [2,4,5,11], and implicit feedback features from previous presentations of the vertical [11,23]. Model tuning and evaluation is typically done with respect to editorial relevance judgements [2,3,4,5,19] or, in a production environment, with respect to user-generated clicks and skips [11,23]. In the first case, users do not actively participate in the evaluation.…”

“…Most published research in aggregated search has focused on automatic methods for predicting which verticals to present (vertical selection) [4,5,11,19] and where in the Web results to present them (vertical presentation) [2,3,23]. Evaluation of these systems has typically been conducted by using editorial vertical relevance judgements as the gold standard [2,3,4,5,19], or by using user-generated clicks on vertical results as a proxy for relevance [11,23].…”

“…Evaluation of these systems has typically been conducted by using editorial vertical relevance judgements as the gold standard [2,3,4,5,19], or by using user-generated clicks on vertical results as a proxy for relevance [11,23]. While these studies have greatly advanced the state of the art in aggregated search techniques, because users are far removed from the evaluation, they have contributed little insight about how users' higher-level objectives influence their engagement with vertical search results.…”

A Methodology for Evaluating Aggregated Search Results

“…Li et al [10] classified queries into two classes of vertical intent (product and job) and evaluated based on precision and recall for each class independently. Diaz [5] focused on predicting when to display news results (always displayed above Web results) and evaluated in terms of correctly predicted clicks and skips.…”

“…Most prior work focuses on vertical selection-the task of predicting which verticals (if any) are relevant to a query [5,10,1,6,2]. The second task of deciding where in the Web results to embed the vertical results has received less attention.…”