Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representations

Lin, Jimmy; Ma, Xueguang; Lin, Sheng-Chieh; Yang, Jheng-Hong; Pradeep, Ronak; Nogueira, Rodrigo

doi:10.48550/arxiv.2102.10073

Cited by 17 publications

(11 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This approach is used as the initial baseline method for text retriever to compare with other techniques. Pyserini is a simple Python package that aids researchers in reproducing their findings by offering excellent first-component document retrieval for multi-component rating systems [13]. Because Pyserini is simple yet effective, it was chosen to be implemented for the text retriever as a component of the QA system.…”

Section: Related Workmentioning

confidence: 99%

XLMRQA: Open-Domain Question Answering on Vietnamese Wikipedia-based Textual Knowledge Source

Nguyen¹,

Do²,

Nguyen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Question answering (QA) is a natural language understanding task within the fields of information retrieval and information extraction that has attracted much attention from the computational linguistics and artificial intelligence research community in recent years because of the strong development of machine reading comprehension-based models. A reader-based QA system is a high-level search engine that can find correct answers to queries or questions in open-domain or domain-specific texts using machine reading comprehension (MRC) techniques. The majority of advancements in data resources and machine-learning approaches in the MRC and QA systems, on the other hand, especially in two resource-rich languages such as English and Chinese. A low-resource language like Vietnamese has witnessed a scarcity of research on QA systems. This paper presents XLMRQA, the first Vietnamese QA system using a supervised transformer-based reader on the Wikipedia-based textual knowledge source (using the UIT-ViQuAD corpus), outperforming the two robust QA systems using deep neural network models: DrQA and BERTserini with 24.46% and 6.28%, respectively. From the results obtained on the three systems, we analyze the influence of question types on the performance of the QA systems.

show abstract

Section: Related Workmentioning

confidence: 99%

XLMRQA: Open-Domain Question Answering on Vietnamese Wikipedia-based Textual Knowledge Source

Nguyen¹,

Do²,

Nguyen³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For sparse retrieval methods, we adpot the pyseirni [32] tool for experiments. For dense retrieval methods, we mainly focus on the DPR architecture.…”

Section: Implementation Detailsmentioning

confidence: 99%

Multi-CPR: A Multi Domain Chinese Dataset for Passage Retrieval

Long¹,

Gao²,

Zou³

et al. 2022

Preprint

View full text Add to dashboard Cite

Passage retrieval is a fundamental task in information retrieval (IR) research, which has drawn much attention recently. In English field, the availability of large-scale annotated dataset (e.g, MS MARCO) and the emergence of deep pre-trained language models (e.g, BERT) have resulted in a substantial improvement of existing passage retrieval systems. However, in Chinese field, especially for specific domain, passage retrieval systems are still immature due to quality-annotated dataset being limited by scale. Therefore, in this paper, we present a novel multi-domain Chinese dataset for passage retrieval (Multi-CPR). The dataset is collected from three different domains, including E-commerce, Entertainment video and Medical. Each dataset contains millions of passages and a certain amount of human annotated query-passage related pairs. We implement various representative passage retrieval methods as baselines. We find that the performance of retrieval models trained on dataset from general domain will inevitably decrease on specific domain. Nevertheless, passage retrieval system built on in-domain annotated dataset can achieve significant improvement, which indeed demonstrates the necessity of domain labeled data for further optimization. We hope the release of the Multi-CPR dataset could benchmark Chinese passage retrieval task in specific domain and also make advances for future studies.

show abstract

“…Inferencing To reproduce the ANCE-PRF results, the authors have provided us with a model checkpoint of PRF depth 3. Since there is no inference code available from the original authors, we utilise the open source IR toolkit Pyserini 4 [16], which has already implemented the ANCE dense retriever, by introducing a second round of ANCE retrieval with the ANCE-PRF model checkpoint. During the inference time, the document index is the same for both the first round ANCE retrieval and the second round ANCE-PRF retrieval.…”

Section: Inferencing and Trainingmentioning

confidence: 99%

Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback: A Reproducibility Study

Li¹,

Zhuang²,

Mourad³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Pseudo-Relevance Feedback (PRF) utilises the relevance signals from the top-k passages from the first round of retrieval to perform a second round of retrieval aiming to improve search effectiveness. A recent research direction has been the study and development of PRF methods for deep language models based rankers, and in particular in the context of dense retrievers. Dense retrievers, compared to more complex neural rankers, provide a trade off between effectiveness, which is often reduced compared to more complex neural rankers, and query latency, which also is reduced making the retrieval pipeline more efficient. The introduction of PRF methods for dense retrievers has been motivated as an attempt to further improve their effectiveness.

show abstract

Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representations

Cited by 17 publications

References 28 publications

XLMRQA: Open-Domain Question Answering on Vietnamese Wikipedia-based Textual Knowledge Source

XLMRQA: Open-Domain Question Answering on Vietnamese Wikipedia-based Textual Knowledge Source

Multi-CPR: A Multi Domain Chinese Dataset for Passage Retrieval

Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback: A Reproducibility Study

Contact Info

Product

Resources

About