Sentence-aware Contrastive Learning for Open-Domain Passage Retrieval

Bo-hong, WU; Zhang, Zhuosheng; Wang, Jin‐Yuan; Zhang, Hai

doi:10.18653/v1/2022.acl-long.76

Cited by 8 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Distant supervision has been used to train MRC models in low-resource settings, and two main kinds of approaches have been proposed to address the mislabeling problem: (1) filtering noisy labels, and (2) modeling answer spans as latent variables. The noise filtering approaches learn to score and rank DS instances based on answer span positions Tay et al, 2018;Swayamdipta et al, 2018;Clark and Gardner, 2018;Lin et al, 2018;Joshi et al, 2017;Chen et al, 2017), question-passage similarities (Hong et al, 2022;Qin et al, 2021;Shao et al, 2021;Deng et al, 2021) and model confidences Zhu et al, 2022). The latent variable-based approaches jointly train MRC models and identify correct answer spans using hard-EM algorithms (Zhao et al, 2021;Min et al, 2019;Cheng et al, 2020).…”

Section: Relate Workmentioning

confidence: 99%

Contrastive Distant Supervision for Debiased and Denoised Machine Reading Comprehension

Bian,

Lin,

Han

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Distant Supervision (DS) is a promising learning approach for MRC by leveraging easilyobtained question-answer pairs. Unfortunately, the heuristically annotated dataset will inevitably lead to mislabeled instances, resulting in answer bias and context noise problems. To learn debiased and denoised MRC models, this paper proposes the Contrastive Distant Supervision algorithm -CDS, which can learn to distinguish confusing and noisy instances via confidence-aware contrastive learning. Specifically, to eliminate answer bias, CDS samples counterfactual negative instances, which ensures that MRC models must take both answer information and question-context interaction into consideration. To denoise distantly annotated contexts, CDS samples confusing negative instances to increase the margin between correct and mislabeled instances. We further propose a confidence-aware contrastive loss to model and leverage the uncertainty of all DS instances during learning. Experimental results show that CDS is effective and can even outperform supervised MRC models without manual annotations.

show abstract

Section: Relate Workmentioning

confidence: 99%

Contrastive Distant Supervision for Debiased and Denoised Machine Reading Comprehension

Bian,

Lin,

Han

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

show abstract

“…Nous verrons à la section suivante que les tâches de pré-entraînement proposées par et Ram et al (2022) génèrent plusieurs pseudo-questions à partir du même passage, en accord avec ces résultats. Ces travaux sont également liés à ceux de Zhang et al (2022) et Hong et al (2022, qui produisent plusieurs représentations par passage. Hong et al (2022), en particulier, démontrent une meilleure robustesse de leur modèle au changement de domaine.…”

Section: Recherche D'information 421 Recherche Neuronale : Dense Ou P...unclassified

“…Ces travaux sont également liés à ceux de Zhang et al (2022) et Hong et al (2022, qui produisent plusieurs représentations par passage. Hong et al (2022), en particulier, démontrent une meilleure robustesse de leur modèle au changement de domaine.…”

Section: Recherche D'information 421 Recherche Neuronale : Dense Ou P...unclassified

Knowledge-based Visual Question Answering about Named Entities

Lerner

2023

SIGIR Forum

View full text Add to dashboard Cite

This thesis is positioned at the intersection of several research fields, Natural Language Processing, Information Retrieval (IR) and Computer Vision, which have unified around representation learning and pre-training methods. We have defined and studied a new multimodal task: Knowledge-based Visual Question Answering about Named Entities (KVQAE). We were particularly interested in cross-modal interactions and different ways of representing named entities. We also focused on data used to train and, more importantly, evaluate Question Answering systems through different metrics. We annotated a dataset for this purpose, the first in KVQAE comprising various types of entities. We also defined an experimental framework for dealing with KVQAE in two stages through an unstructured knowledge base and identified IR as the main bottleneck of KVQAE, especially for questions about non-person entities. To improve the IR stage, we studied different multimodal fusion methods, which are pre-trained through an original task: the Multimodal Inverse Cloze Task. We found that these models leveraged a cross-modal interaction that we had not originally considered, and which may address the heterogeneity of visual representations of named entities. These results were strengthened by a study of the CLIP model, which allows this cross-modal interaction to be modeled directly. Awarded by : Université Paris-Saclay, Orsay, France on 8 November 2023. Supervised by : Olivier Ferret and Camille Guinaudeau. Available at : https://www.theses.fr/s247993.

show abstract

WSRR: Weighted Rank-Relevance Sampling for Dense Text Retrieval

Hambarde,

Proença

2023

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

Sentence-aware Contrastive Learning for Open-Domain Passage Retrieval

Cited by 8 publications

References 15 publications

Contrastive Distant Supervision for Debiased and Denoised Machine Reading Comprehension

Contrastive Distant Supervision for Debiased and Denoised Machine Reading Comprehension

Knowledge-based Visual Question Answering about Named Entities

WSRR: Weighted Rank-Relevance Sampling for Dense Text Retrieval

Contact Info

Product

Resources

About