Multi-Task Learning for Email Search Ranking with Auxiliary Query Clustering

Abstract

User information needs vary significantly across different tasks, and therefore their queries will also differ considerably in their expressiveness and semantics. Many studies have been proposed to model such query diversity by obtaining query types and building query-dependent ranking models. These studies typically require either a labeled query dataset or clicks from multiple users aggregated over the same document. These techniques, however, are not applicable when manual query labeling is not viable, and aggregated clicks are unavailable due to the private nature of the document collection, e.g., in email search scenarios. In this paper, we study how to obtain query type in an unsupervised fashion and how to incorporate this information into query-dependent ranking models. We first develop a hierarchical clustering algorithm based on truncated SVD and varimax rotation to obtain coarse-to-fine query types. Then, we study three querydependent ranking models, including two neural models that leverage query type information as additional features, and one novel multi-task neural model that views query type as the label for the auxiliary query cluster prediction task. This multi-task model is trained to simultaneously rank documents and predict query types. Our experiments on tens of millions of real-world email search queries demonstrate that the proposed multi-task model can significantly outperform the baseline neural ranking models, which either do not incorporate query type information or just simply feed query type as an additional feature.

Separate and Attend in Personal Email Search

“…Ai et al [2] conducted a thorough survey of search intent by analyzing user logs of email search. Shen et al [39] categorized email search queries into different clusters before adding the query cluster information to improve email ranking.…”

“…Recently, deep neural networks (DNNs) have shown great success in learning-to-rank tasks. They significantly improve the performance of search engines in the presence of large-scale query logs in both web search [19] and email settings [39,45,51]. The advantages of DNNs over traditional models are mainly two-fold: (1) DNNs have strong power to learn embedded representations from sparse features, including words [33] and characters [6].…”

“…However, there have been few efforts that study how to effectively exploit both dense and sparse features in the learningto-rank setting, probably because a natural approach exists-simply concatenating dense features with embedded sparse features and feeding them into the DNNs. Indeed, many previous deep neural email search models use direct concatenation of dense features with embedded sparse features [15,38,39,45].…”

Domain Adaptation for Enterprise Email Search

“…Enterprise search is also closely related to personal search (e.g., email search), as both deal with searching in private or access controlled corpora [2,8,12,20,26,39,44]. Even though some success has been found using time-based approaches for personal search [12], relevance-based ranking arising from learning-to-rank deep neural network models has become increasingly popular [39,46] as the sizes of private corpora increase [20]. However, to the best of our knowledge, our work is the first study on applying deep neural networks specifically in the enterprise search setting.…”

“…To preserve privacy, the inputs are k-anonymized and only query and document n-grams that are frequent in the entire corpus are retained. For more details about the specific features and the anonymization process used in a similar setting see [39,46]. The query features and sparse document features are passed through a single embedding layer, whereas the dense features are left as they are.…”