GLOW : Global Weighted Self-Attention Network for Web Search

Shan, Xuan; Liu, Chuanjie; Xia, Yiqian; Chen, Qi; Zhang, Yusi; Ding, Kaize; Liang, Yilong; Luo, Angen; Luo, Yu

doi:10.1109/bigdata52589.2021.9671546

Cited by 7 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since embedding-based retrieval solves the above three challenges, it has been widely used in modern information systems to facilitate new state-of-the-art retrieval quality and performance. Numerous prior studies have concentrated on deep embedding models, from DSSM [21], CDSSM [46], LSTM-RNN [38], and ARC-I [20] to transformer-based embedding models [10,16,39,40,45,53,54]. They have shown impressive gains with brute-force nearest neighbor embedding search on some small datasets as compared with traditional keyword matching.…”

Section: Background and Related Work 21 Web Scale Information Retrievalmentioning

confidence: 99%

MS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels

Chen,

Geng,

Rosset

et al. 2024

Companion Proceedings of the ACM Web Conference 2024

View full text Add to dashboard Cite

Recent breakthroughs in large models have highlighted the critical significance of data scale, labels and modals. In this paper, we introduce MS MARCO Web Search, the first large-scale information-rich web dataset, featuring millions of real clicked query-document labels. This dataset closely mimics real-world web document and query distribution, provides rich information for various kinds of downstream tasks and encourages research in various areas, such as generic end-to-end neural indexer models, generic embedding models, and next generation information access system with large language models. MS MARCO Web Search offers a retrieval benchmark with three web retrieval challenge tasks that demands innovations in both machine learning and information retrieval system research domains. As the first dataset that meets large, real and rich data requirements, MS MARCO Web Search paves the way for future advancements in AI and system research. MS MARCO Web Search dataset is available at: https://github.com/microsoft/MS-MARCO-Web-Search.

show abstract

Section: Background and Related Work 21 Web Scale Information Retrievalmentioning

confidence: 99%

MS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels

Chen,

Geng,

Rosset

et al. 2024

Companion Proceedings of the ACM Web Conference 2024

View full text Add to dashboard Cite

show abstract

“…Zhan et al [96] presented an efficient technique for training dense retrieval models using a combination of hard and soft negative sampling. Meanwhile, Shan et al [97] employed a global weighted self-attention network for web search. Other researchers have explored optimization techniques and innovative methodologies to enhance the performance of dense retrieval models in various scenarios [98], [99], [100], [101], [102], [103].…”

Section: ) Approximate Nearest Neighbor Search and Negative Contrasti...mentioning

confidence: 99%

Information Retrieval: Recent Advances and Beyond

Hambarde

Proença

2023

IEEE Access

View full text Add to dashboard Cite

This paper provides an extensive and thorough overview of the models and techniques utilized in the first and second stages of the typical information retrieval processing chain. Our discussion encompasses the current state-of-the-art models, covering a wide range of methods and approaches in the field of information retrieval. We delve into the historical development of these models, analyze the key advancements and breakthroughs, and address the challenges and limitations faced by researchers and practitioners in the domain. By offering a comprehensive understanding of the field, this survey is a valuable resource for researchers, practitioners, and newcomers to the information retrieval domain, fostering knowledge growth, innovation, and the development of novel ideas and techniques.INDEX TERMS First-stage retrieval, information retrieval, second-stage retrieval.

show abstract

“…Even so, combinations of vectorizing and symbolic methods are not often discussed. BISON (Shan et al , 2020) adds a new layer of information through an attention model to bring different weights to each of the tokens by using the BM25 measure. Although remaining in a vector representation while processing queries and online documents generates some of the problems mentioned above, this approach exploits the advantages of both methods: word importance in the corpus and documents, taking into account word context, text dimension reduction, etc.…”

Section: Related Workmentioning

confidence: 99%

LoGE: an unsupervised local-global document extension generation in information retrieval for long documents

Ayoub,

Rodrigues,

Travers

2023

IJWIS

View full text Add to dashboard Cite

Purpose This paper aims to manage the word gap in information retrieval (IR) especially for long documents belonging to specific domains. In fact, with the continuous growth of text data that modern IR systems have to manage, existing solutions are needed to efficiently find the best set of documents for a given request. The words used to describe a query can differ from those used in related documents. Despite meaning closeness, nonoverlapping words are challenging for IR systems. This word gap becomes significant for long documents from specific domains. Design/methodology/approach To generate new words for a document, a deep learning (DL) masked language model is used to infer related words. Used DL models are pretrained on massive text data and carry common or specific domain knowledge to propose a better document representation. Findings The authors evaluate the approach of this study on specific IR domains with long documents to show the genericity of the proposed model and achieve encouraging results. Originality/value In this paper, to the best of the authors’ knowledge, an original unsupervised and modular IR system based on recent DL methods is introduced.

show abstract

GLOW : Global Weighted Self-Attention Network for Web Search

Cited by 7 publications

References 15 publications

MS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels

MS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels

Information Retrieval: Recent Advances and Beyond

LoGE: an unsupervised local-global document extension generation in information retrieval for long documents

Contact Info

Product

Resources

About