Learning Maximal Marginal Relevance Model via Directly Optimizing Diversity Evaluation Measures

Xia, Long; Xu, Jun; Lan, Yanyan; Guo, Jiafeng; Cheng, Xueqi

doi:10.1145/2766462.2767710

Cited by 56 publications

(39 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Learn to match Prior state-of-the-art methods for diversifying search results include the Relational Learning-to-Rank framework (R-LTR) ) and the Perceptron Algorithm using Measures as Margins (PAMM) (Xia et al 2015). These prior methods either use a heuristic ranking model based on a predefined document similarity function, or they automatically learn a ranking model from predefined novelty features often based on cosine similarity.…”

Section: Learnmentioning

confidence: 99%

Neural information retrieval: at the end of the early years

et al. 2017

View full text Add to dashboard Cite

A recent ''third wave'' of neural network (NN) approaches now delivers state-ofthe-art performance in many machine learning tasks, spanning speech recognition, computer vision, and natural language processing. Because these modern NNs often comprise multiple interconnected layers, work in this area is often referred to as deep learning. Recent years have witnessed an explosive growth of research into NN-based approaches to information retrieval (IR). A significant body of work has now been created. In this paper,

show abstract

Section: Learnmentioning

confidence: 99%

Neural information retrieval: at the end of the early years

et al. 2017

View full text Add to dashboard Cite

show abstract

“…With different definitions of the objective functions and optimization techniques, different diverse ranking algorithms have been proposed [21,22,24]. Xia et al [21] learn a maximal marginal relevance model via directly optimizing diversity evaluation measures. The authors in [22] utilize the neural tensor network to model the novelty relations.…”

Section: Search Results Diversificationmentioning

confidence: 99%

Novel Approaches to Accelerating the Convergence Rate of Markov Decision Process for Search Result Diversification

Liu

Tang

et al. 2018

Database Systems for Advanced Applications

View full text Add to dashboard Cite

Recently, some studies have utilized the Markov Decision Process for diversifying (MDP-DIV) the search results in information retrieval. Though promising performances can be delivered, MDP-DIV suffers from a very slow convergence, which hinders its usability in real applications. In this paper, we aim to promote the performance of MDP-DIV by speeding up the convergence rate without much accuracy sacrifice. The slow convergence is incurred by two main reasons: the large action space and data scarcity. On the one hand, the sequential decision making at each position needs to evaluate the query-document relevance for all the candidate set, which results in a huge searching space for MDP; on the other hand, due to the data scarcity, the agent has to proceed more "trial and error" interactions with the environment. To tackle this problem, we propose MDP-DIV-kNN and MDP-DIV-NTN methods. The MDP-DIV-kNN method adopts a k nearest neighbor strategy, i.e., discarding the k nearest neighbors of the recently-selected action (document), to reduce the diversification searching space. The MDP-DIV-NTN employs a pre-trained diversification neural tensor network (NTN-DIV) as the evaluation model, and combines the results with MDP to produce the final ranking solution. The experiment results demonstrate that the two proposed methods indeed accelerate the convergence rate of the MDP-DIV, which is 3x faster, while the accuracies produced barely degrade, or even are better.

show abstract

“…Well-known examples include xQuAD [37], RxQuAD [41], IA-select [2], PM-2 [13], and learning models for diversification [25,27,45]. Instead of modeling a set of aspects implicitly, these algorithms obtain a set of aspects either manually, for example, from aspect descriptions [9,11], or they create them directly from, for example, suggested queries generated by commercial search engines [13,37], or predefined aspect categories [40] or directly utilize the human judged labels of aspects for learning [25,27,45].…”

Section: Search Results Diversificationmentioning

confidence: 99%

Search Result Diversification in Short Text Streams

Liang

Yılmaz

Shen

et al. 2017

ACM Trans. Inf. Syst.

View full text Add to dashboard Cite

We consider the problem of search result diversification for streams of short texts. Diversifying search results in short text streams is more challenging than in the case of long documents, as it is difficult to capture the latent topics of short documents. To capture the changes of topics and the probabilities of documents for a given query at a specific time in a short text stream, we propose a dynamic Dirichlet multinomial mixture topic model, called D2M3, as well as a Gibbs sampling algorithm for the inference. We also propose a streaming diversification algorithm, SDA, that integrates the information captured by D2M3 with our proposed modified version of the PM-2 (Proportionality-based diversification Methodsecond version) diversification algorithm. We conduct experiments on a Twitter dataset and find that SDA statistically significantly outperforms state-of-the-art non-streaming retrieval methods, plain streaming retrieval methods, as well as streaming diversification methods that use other dynamic topic models.

show abstract

Learning Maximal Marginal Relevance Model via Directly Optimizing Diversity Evaluation Measures

Cited by 56 publications

References 29 publications

Neural information retrieval: at the end of the early years

Neural information retrieval: at the end of the early years

Novel Approaches to Accelerating the Convergence Rate of Markov Decision Process for Search Result Diversification

Search Result Diversification in Short Text Streams

Contact Info

Product

Resources

About