Generalised Unsupervised Domain Adaptation of Neural Machine Translation with Cross-Lingual Data Selection

Vu, Thuy-Trang; He, Xuanli; Phung, Dinh; Haffari, Gholamreza

doi:10.18653/v1/2021.emnlp-main.268

Cited by 5 publications

(4 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In summary, the combination of different methods is the key to obtaining high-quality alignments, see for example Fei, Zhang, and Ji (2020), Steingrimsson, Loftsson, and Way (2021), Vu et al (2021). It is a crucial point for the creation of interlinear glossed biblical texts to really understand the detailed concepts of the languages and either use large training corpora and supervise the results of these methods, for example, by manually curating the texts.…”

Section: Related Workmentioning

confidence: 99%

Automated annotation of parallel bible corpora with cross-lingual semantic concordance

Dörpinghaus

2024

Nat. Lang. Eng.

View full text Add to dashboard Cite

Here we present an improved approach for automated annotation of New Testament corpora with cross-lingual semantic concordance based on Strong’s numbers. Based on already annotated texts, they provide references to the original Greek words. Since scientific editions and translations of biblical texts are often not available for scientific purposes and are rarely freely available, there is a lack of up-to-date training data. In addition, since annotation, curation, and quality control of alignments between these texts are expensive, there is a lack of available biblical resources for scholars. We present two improved approaches to the problem, based on dictionaries and already annotated biblical texts. We provide a detailed evaluation of annotated and unannotated translations. We also discuss a proof of concept based on English and German New Testament translations. The results presented in this paper are novel and, to our knowledge, unique. They show promising performance, although further research is needed.

show abstract

Section: Related Workmentioning

confidence: 99%

Automated annotation of parallel bible corpora with cross-lingual semantic concordance

Dörpinghaus

2024

Nat. Lang. Eng.

View full text Add to dashboard Cite

show abstract

“…Cross-Lingual Data Selection Vu et al (2021) proposed a generalized unsupervised domain adaptation technique (GUDA) for NMT where only monolingual data from either the source or target language is available in the new domain. A cross-lingual data selection method is introduced to select relevant in-domain sentences from a large monolingual corpus for the language without in-domain data.…”

Section: Unsupervised Domain Adaptation Withmentioning

confidence: 99%

Reference Free Domain Adaptation for Translation of Noisy Questions with Question Specific Rewards

Gain,

Appicharla,

Chennabasavaraj

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Community Question-Answering (CQA) portals serve as a valuable tool for helping users within an organization. However, making them accessible to non-English-speaking users continues to be a challenge. Translating questions can broaden the community's reach, benefiting individuals with similar inquiries in various languages. Translating questions using Neural Machine Translation (NMT) poses more challenges, especially in noisy environments, where the grammatical correctness of the questions is not monitored. These questions may be phrased as statements by non-native speakers, with incorrect subject-verb order and sometimes even missing question marks. Creating a synthetic parallel corpus from such data is also difficult due to its noisy nature. To address this issue, we propose a training methodology that fine-tunes the NMT system only using source-side data. Our approach balances adequacy and fluency by utilizing a loss function that combines BERTScore and Masked Language Model (MLM) Score. Our method surpasses the conventional Maximum Likelihood Estimation (MLE) based fine-tuning approach, which relies on synthetic target data, by achieving a 1.9 BLEU score improvement. Our model exhibits robustness while we add noise to our baseline, and still achieve 1.1 BLEU improvement and large improvements on TER and BLEURT metrics. Our proposed methodology is model-agnostic and is only necessary during the training phase. We make the codes and datasets publicly available at https: //www.iitp.ac.in/~ai-nlp-ml/ resources.html#DomainAdapt for facilitating further research.

show abstract

“…This suggests a promising avenue for future research on using synthetically generate monolingual data to improve MT for specialized domains where even monolingual data is scarce. Furthermore, Vu et al (2021a) suggest that one can leverage a retrieval-based to obtain monolingual sentences from the generic data stores. This retrieved monolingual data is then employed to improve the translation quality in a domain adaptation setting.…”

Section: Limitationsmentioning

confidence: 99%

Generate, Annotate, and Learn: NLP with Synthetic Text

Nassar

Kiros

et al. 2022

Transactions of the Association for Computational Linguistics

View full text Add to dashboard Cite

This paper studies the use of language models as a source of synthetic unlabeled text for NLP. We formulate a general framework called “generate, annotate, and learn (GAL)” to take advantage of synthetic text within knowledge distillation, self-training, and few-shot learning applications. To generate high-quality task-specific text, we either fine-tune LMs on inputs from the task of interest, or prompt large LMs with few examples. We use the best available classifier to annotate synthetic text with soft pseudo labels for knowledge distillation and self-training, and use LMs to obtain hard labels for few-shot learning. We train new supervised models on the combination of labeled and pseudo-labeled data, which results in significant gains across several applications. We investigate key components of GAL and present theoretical and empirical arguments against the use of class-conditional LMs to generate synthetic labeled text instead of unlabeled text. GAL achieves new state-of-the-art knowledge distillation results for 6-layer transformers on the GLUE leaderboard.

show abstract

Generalised Unsupervised Domain Adaptation of Neural Machine Translation with Cross-Lingual Data Selection

Cited by 5 publications

References 30 publications

Automated annotation of parallel bible corpora with cross-lingual semantic concordance

Automated annotation of parallel bible corpora with cross-lingual semantic concordance

Reference Free Domain Adaptation for Translation of Noisy Questions with Question Specific Rewards

Generate, Annotate, and Learn: NLP with Synthetic Text

Contact Info

Product

Resources

About