Improving Low-Resource Cross-lingual Document Retrieval by Reranking with Deep Bilingual Representations

Zhang, Rui; Westerfield, Caitlin; Shim, Sungrok; Bingham, Garrett; Fabbri, Alexander R.; Verma, Neha; Hu, William; Radev, Dragomir

doi:10.48550/arxiv.1906.03492

Cited by 5 publications

(1 citation statement)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the retrieve-and-rerank framework has been adapted and explored in cross-language information retrieval (CLIR) [37,11,38,3,36], most approaches translate queries into the language of the documents and perform monolingual retrieval [28,29]. Dense retrieval models, on the other hand, remain under-explored in CLIR.…”

Section: Introductionmentioning

confidence: 99%

Transfer Learning Approaches for Building Cross-Language Dense Retrieval Models

Nair¹,

Yang²,

Lawrie³

et al. 2022

Preprint

View full text Add to dashboard Cite

The advent of transformer-based models such as BERT has led to the rise of neural ranking models. These models have improved the effectiveness of retrieval systems well beyond that of lexical term matching models such as BM25. While monolingual retrieval tasks have benefited from large-scale training collections such as MS MARCO and advances in neural architectures, cross-language retrieval tasks have fallen behind these advancements. This paper introduces ColBERT-X, a generalization of the ColBERT multi-representation dense retrieval model that uses the XLM-RoBERTa (XLM-R) encoder to support cross-language information retrieval (CLIR). ColBERT-X can be trained in two ways. In zero-shot training, the system is trained on the English MS MARCO collection, relying on the XLM-R encoder for cross-language mappings. In translate-train, the system is trained on the MS MARCO English queries coupled with machine translations of the associated MS MARCO passages. Results on ad hoc document ranking tasks in several languages demonstrate substantial and statistically significant improvements of these trained dense retrieval models over traditional lexical CLIR baselines.

show abstract

Section: Introductionmentioning

confidence: 99%

Transfer Learning Approaches for Building Cross-Language Dense Retrieval Models

Nair¹,

Yang²,

Lawrie³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

CLIRMatrix: A massively large collection of bilingual and multilingual datasets for Cross-Lingual Information Retrieval

Sun¹,

Duh²

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

We present CLIRMatrix, a massively large collection of bilingual and multilingual datasets for Cross-Lingual Information Retrieval extracted automatically from Wikipedia. CLIR-Matrix comprises (1) BI-139, a bilingual dataset of queries in one language matched with relevant documents in another language for 139×138=19,182 language pairs, and(2) MULTI-8, a multilingual dataset of queries and documents jointly aligned in 8 different languages. In total, we mined 49 million unique queries and 34 billion (query, document, label) triplets, making it the largest and most comprehensive CLIR dataset to date. This collection is intended to support research in end-to-end neural information retrieval and is publicly available at https: //github.com/ssun32/CLIRMatrix. We provide baseline neural model results on BI-139, and evaluate MULTI-8 in both singlelanguage retrieval and mix-language retrieval settings.

show abstract

Cross-language Sentence Selection via Data Augmentation and Rationale Training

Chen¹,

Kedzie²,

Nair³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

This paper proposes an approach to crosslanguage sentence selection in a low-resource setting. It uses data augmentation and negative sampling techniques on noisy parallel sentence data to directly learn a cross-lingual embedding-based query relevance model. Results show that this approach performs as well as or better than multiple state-of-theart machine translation + monolingual retrieval systems trained on the same parallel data. Moreover, when a rationale training secondary objective is applied to encourage the model to match word alignment hints from a phrase-based statistical machine translation model, consistent improvements are seen across three language pairs (English-Somali, English-Swahili and English-Tagalog) over a variety of state-of-the-art baselines.

show abstract

Improving Low-Resource Cross-lingual Document Retrieval by Reranking with Deep Bilingual Representations

Cited by 5 publications

References 25 publications

Transfer Learning Approaches for Building Cross-Language Dense Retrieval Models

Transfer Learning Approaches for Building Cross-Language Dense Retrieval Models

CLIRMatrix: A massively large collection of bilingual and multilingual datasets for Cross-Lingual Information Retrieval

Cross-language Sentence Selection via Data Augmentation and Rationale Training

Contact Info

Product

Resources

About