BERT-QPP: Contextualized Pre-trained transformers for Query Performance Prediction

Arabzadeh, Negar; Khodabakhsh, Maryam; Bagheri, Ebrahim

doi:10.1145/3459637.3482063

Cited by 24 publications

(25 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, they do not necessarily agree on the predicted performance according to different queries, corpora, or retrieval methods. This observation has been made on different SOTA QPP methods and various well-known corpora such as TREC corpora or MS MARCO and their associated query set [8,16,34]. Thus, we conclude that the level of agreement could strengthen our confidence in the query performance prediction.…”

Section: Resultssupporting

confidence: 63%

Supporting Complex Information-Seeking Tasks with Implicit Constraints

Ahmadvand¹,

Arabzadeh²,

Kiseleva³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Current interactive systems with natural language interface lack an ability to understand a complex information-seeking request which expresses several implicit constraints at once, and there is no prior information about user preferences, e.g., 'find hiking trails around San Francisco which are accessible with toddlers and have beautiful scenery in summer', where output is a list of possible suggestions for users to start their exploration. In such scenarios, the user requests can be issued at once in the form of a complex and long query, unlike conversational and exploratory search models that require short utterances or queries where they often require to be fed into the system step by step. This advancement provides the final user more flexibility and precision in expressing their intent through the search process. Such systems are inherently helpful for day-today user tasks requiring planning that are usually time-consuming, sometimes tricky, and cognitively taxing. We have designed and deployed a platform to collect the data from approaching such complex interactive systems.In this paper, we propose an Interactive Agent (IA) that allows intricately refined user requests by making it complete, which should lead to better retrieval. To demonstrate the performance of the proposed modeling paradigm, we have adopted various pre-retrieval metrics that capture the extent to which guided interactions with our system yield better retrieval results. Through extensive experimentation, we demonstrated that our method significantly outperforms several robust baselines.

show abstract

Section: Resultssupporting

confidence: 63%

Supporting Complex Information-Seeking Tasks with Implicit Constraints

Ahmadvand¹,

Arabzadeh²,

Kiseleva³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Encoding query-document pairs. Following Arabzadeh et al [4], we first encode each query-document pair with BERT. As documents are frequently long enough to exceed BERT's 512 token limit, similar to Co-BERT [8], we split long texts into equal-sized passages.…”

Section: Methodsmentioning

confidence: 99%

“…With the recent development deep learning techniques, NeuralQPP [51] achieves promising results by training a three-components deep network under weak supervision of existing methods. Recently, while NQA-QPP [22] uses BERT to generate contextualized embedding for QPP in non-factoid question answering, BERT-QPP [4] directly applies BERT with pointwise learning in the prediction task, outperforming previous methods on the MS MARCO dev set [30] and TREC Deep Learning track query sets [13,14]. Groupwise Ranking.…”

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…Indeed, recent results demonstrate that BERT effectively improves the performance of post-retrieval QPP [4,22]. For instance, training with a large number of sparse-labeled queries and their highest-ranked documents, BERT-QPP [4] examines the effectiveness of BERT on the MS MARCO [30] and TREC DL [13,14] datasets, by pointwise modeling of query-document pairs. Beyond learning from single query-document pairs, the groupwise methods have achieved superior performance on both learning-to-rank [1,2,33,32] and BERT re-ranking [8] benchmarks.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Groupwise Query Performance Prediction with BERT

Chen¹,

He²,

Sun³

2022

Preprint

View full text Add to dashboard Cite

While large-scale pre-trained language models like BERT have advanced the state-of-the-art in IR, its application in query performance prediction (QPP) is so far based on pointwise modeling of individual queries. Meanwhile, recent studies suggest that the cross-attention modeling of a group of documents can effectively boost performances for both learning-to-rank algorithms and BERT-based re-ranking. To this end, a BERT-based groupwise QPP model is proposed, in which the ranking contexts of a list of queries are jointly modeled to predict the relative performance of individual queries. Extensive experiments on three standard TREC collections showcase effectiveness of our approach.

show abstract

Groupwise Query Performance Prediction with BERT

Chen

Sun

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

BERT-QPP: Contextualized Pre-trained transformers for Query Performance Prediction

Cited by 24 publications

References 26 publications

Supporting Complex Information-Seeking Tasks with Implicit Constraints

Supporting Complex Information-Seeking Tasks with Implicit Constraints

Groupwise Query Performance Prediction with BERT

Groupwise Query Performance Prediction with BERT

Contact Info

Product

Resources

About