A Hierarchical Recurrent Encoder-Decoder for Generative Context-Aware Query Suggestion

Abstract

Users may strive to formulate an adequate textual query for their information need. Search engines assist the users by presenting query suggestions. To preserve the original search intent, suggestions should be context-aware and account for the previous queries issued by the user. Achieving context awareness is challenging due to data sparsity. We present a probabilistic suggestion model that is able to account for sequences of previous queries of arbitrary lengths. Our novel hierarchical recurrent encoder-decoder architecture allows the model to be sensitive to the order of queries in the context while avoiding data sparsity. Additionally, our model can suggest for rare, or long-tail, queries. The produced suggestions are synthetic and are sampled one word at a time, using computationally cheap decoding techniques. This is in contrast to current synthetic suggestion models relying upon machine learning pipelines and hand-engineered feature sets. Results show that it outperforms existing context-aware approaches in a next query prediction setting. In addition to query suggestion, our model is general enough to be used in a variety of other applications.

Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses

“…The hierarchical RNN encoder in our model consists of two layers of RNNs (El Hihi and Bengio, 1995;Sordoni et al, 2015a). The lower-level RNN, the utterance-level encoder, takes as input words from the dialogue, and produces a vector output at the end of each utterance.…”

Exploiting Cross-Sentence Context for Neural Machine Translation

“…Sordoni et al (2015) use HRED to summarize a single representation from both the current and previous sentences, which limits itself to (1) it is only applicable to encoder-decoder framework without attention model, (2) the representation can only be used to initialize decoder. In contrast, we use HRED to summarize the previous sentences alone, which provides additional cross-sentence context for NMT.…”

“…1 In this paper, we propose a cross-sentence context-aware NMT model, which considers the influence of previous source sentences in the same document. 2 Specifically, we employ a hierarchy of Recurrent Neural Networks (RNNs) to summarize the cross-sentence context from source-side previous sentences, which deploys an additional documentlevel RNN on top of the sentence-level RNN encoder (Sordoni et al, 2015). After obtaining the global context, we design several strategies to integrate it into NMT to translate the current sentence:…”

Towards Automatic Generation of Product Reviews from Aspect-Sentiment Scores

“…Traditional way for text generation (Genest and Lapalme, 2012;Yan et al, 2011) mainly focus on grammars, templates, and so on. But it is usually complicated to make every part of the system work and cooperate perfectly following the traditional techniques, while end-to-end generation systems nowadays, like the ones within encoder-decoder framework Sordoni et al, 2015), have distinct architectures and achieve promising performances.…”