CUED@WMT19:EWC&amp;LMs

Stahlberg, Felix; Saunders, Danielle; Gispert, Adrià de; Byrne, Bill

doi:10.18653/v1/w19-5340

Cited by 12 publications

(6 citation statements)

References 48 publications

(56 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, it was shown that the shallow fusion approach for sentencelevel NMT can be improved by compensating for the implicitly learned internal language model of the NMT system . Regarding the integration of a document-level LM, earlier approaches simply use the LM for re-ranking the hypothesis of the sentence-level NMT model (Stahlberg et al, 2019;Yu et al, 2020). Several works have proposed to employ a log-linear combination between sentence-level NMT system and document-level LM (Garcia et al, 2019;Jean and Cho, 2020;Sugiyama and Yoshinaga, 2020).…”

Section: Related Workmentioning

confidence: 99%

Document-Level Language Models for Machine Translation

Petrick,

Herold,

Petrushkov

et al. 2023

Proceedings of the Eighth Conference on Machine Translation

View full text Add to dashboard Cite

Despite the known limitations, most machine translation systems today still operate on the sentence-level. One reason for this is, that most parallel training data is only sentencelevel aligned, without document-level meta information available. In this work, we set out to build context-aware translation systems utilizing document-level monolingual data instead. This can be achieved by combining any existing sentence-level translation model with a document-level language model. We improve existing approaches by leveraging recent advancements in model combination. Additionally, we propose novel weighting techniques that make the system combination more flexible and significantly reduce computational overhead. In a comprehensive evaluation on four diverse translation tasks, we show that our extensions improve document-targeted scores substantially and are also computationally more efficient. However, we also find that in most scenarios, back-translation gives even better results, at the cost of having to re-train the translation system. Finally, we explore language model fusion in the light of recent advancements in large language models. Our findings suggest that there might be strong potential in utilizing large language models via model combination.

show abstract

Section: Related Workmentioning

confidence: 99%

Document-Level Language Models for Machine Translation

Petrick,

Herold,

Petrushkov

et al. 2023

Proceedings of the Eighth Conference on Machine Translation

View full text Add to dashboard Cite

show abstract

“…Beam search appears to be more exact, but there is no assurance that it will always lead to an interpretation with a higher or comparable score than greedy decoding. [38] According to Stahlberg and Byrne (2019), beam search has a huge number of searching errors.…”

Section: A Greedy and Beam Searchmentioning

confidence: 99%

“…Stahlberg et al. (2018),[38]Stahlberg and Byrne,[40]Niehues et al (2017), [41][42]Stahlberg et al (2018), and (2019). In particular,[38]Stahlberg and Byrne (2019) showed that the NMT decoding had a substantial number of search mistakes.…”

mentioning

confidence: 99%

An Insight into Neural Machine Translation

Goswami,

Vadivu,

Ram

et al. 2023

Recent Trends in Data Science and Its Applications

View full text Add to dashboard Cite

Automatic language translation between two languages has encountered a quantum leap in perspective as of late in the field of machine learning. The term "neural machine translation" was developed in response to statistical machine translation, which relies on various count-based models and long dominated MT research. In contrast to conventional statistical machine translation, neural machine translation aims to build a single neural network that may be mutually changed to maximize translation efficiency. The current NMT models may be traced to previous versions of the encoderdecoder network family as well as to word and sentence embedding in this study. We will conclude with a succinct outline of recent developments in fields like NMT's bidirectional training (BiT).

show abstract

“…The Cambridge University Engineering Department's system [101] relied on document-level language models to improve the sentence-level NMT system. They modified the Transformer architecture for document-level language modelling by introducing separate attention layers for inter-and intra-sentential context.…”

Section: Shared Tasks In Wmt19 and Wngt19mentioning

confidence: 99%

A Survey on Document-level Neural Machine Translation: Methods and Evaluation

Maruf¹,

Saleh²,

Haffari³

2019

Preprint

View full text Add to dashboard Cite

Machine translation (MT) is an important task in natural language processing (NLP) as it automates the translation process and reduces the reliance on human translators. With the advent of neural networks, the translation quality surpasses that of the translations obtained using statistical techniques. Up until three years ago, all neural translation models translated sentences independently, without incorporating any extra-sentential information. The aim of this paper is to highlight the major works that have been undertaken in the space of documentlevel machine translation before and after the neural revolution so that researchers can recognise where we started from and which direction we are heading in. When talking about the literature in statistical machine translation (SMT), we focus on works which have tried to improve the translation of specific discourse phenomena, while in neural machine translation (NMT), we focus on works which use the wider context explicitly. In addition to this, we also cover the evaluation strategies that have been introduced to account for the improvements in this domain.

show abstract

CUED@WMT19:EWC&LMs

Cited by 12 publications

References 48 publications

Document-Level Language Models for Machine Translation

Document-Level Language Models for Machine Translation

An Insight into Neural Machine Translation

A Survey on Document-level Neural Machine Translation: Methods and Evaluation

Contact Info

Product

Resources

About