Evaluation of Neural Machine Translation for Highly Inflected and Small Languages

Pinnis, Mārcis; Krišlauks, Rihards; Deksne, Daiga; Miks, Toms

doi:10.1007/978-3-319-77116-8_33

Cited by 4 publications

(5 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the automatic metrics have shown not to be sufficient to evaluate MT systems of the two different paradigms (Pinnis et al, 2017a), we also performed (blind) human comparative evaluation of the SMT and NMT system translations. Five professional translators were given the source sentence and translations of two MT systems and asked to select, which system (NMT, SMT, or neither) produces a better translation.…”

Section: Resultsmentioning

confidence: 99%

“…Our previous research in NMT system development has shown that NMT systems are more sensitive to the noise present in the training data (Pinnis et al, 2017a) than SMT systems, therefore, we performed parallel data filtering to reduce potential non-parallelities and the negative effect of noise on the NMT systems. The filtering consisted of the following steps:…”

Section: Data Filteringmentioning

confidence: 99%

See 1 more Smart Citation

Tilde's Machine Translation Systems for WMT 2017

Pinnis¹,

Krišlauks²,

Miks³

et al. 2017

Proceedings of the Second Conference on Machine Translation

Self Cite

View full text Add to dashboard Cite

The paper describes Tilde's EnglishLatvian and Latvian-English machine translation systems for the WMT 2017 shared task in news translation. Both constrained and unconstrained systems are described. Our constrained systems were ranked as the best performing systems according to the automatic evaluation results. The paper gives details to how we pre-processed training data, the NMT system architecture that we used for training the NMT models, the SMT systems and their usage in NMT-SMT hybrid system configurations.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Data Filteringmentioning

confidence: 99%

Tilde's Machine Translation Systems for WMT 2017

Pinnis¹,

Krišlauks²,

Miks³

et al. 2017

Proceedings of the Second Conference on Machine Translation

Self Cite

View full text Add to dashboard Cite

show abstract

“…Moorkens, 2018;Briggs, 2018) or professionals (e.g. Pinnis et al, 2018;López Pereira, 2018;Daems and Macken, 2019). This preference margin was on occasion minimal, however.…”

Section: Perceptionsmentioning

confidence: 97%

“…Bentivogli et al 2016; and inflectional morphology (e.g. Popović, 2017), especially agreement (Klubička et al, 2017), including long distance agreements (Pinnis et al, 2018;Isabelle et al, 2017). However, NMT system performance has been consistently shown to deteriorate with increasing sentence length 9 (e.g.…”

Section: Quality and Human Evaluationmentioning

confidence: 99%

What has changed with neural machine translation? A critical review of human factors

Ragni

Vieira

2021

Perspectives

View full text Add to dashboard Cite

In recent years, the push towards automation and translation productivity led to great efforts dedicated to the development of machine translation (MT) systems. Neural machine translation (NMT) represents the latest of these efforts. In this paper we present a critical review of human factors in NMT research with two goals: to provide a snapshot of research in NMT involving human stakeholders, and to appraise how professional translators have been included in discourses around NMT. We report four key findings. First, from translators' perspective, changes brought about by the neural paradigm are not as much to do with workflows, but rather with the NMT editing process and its specifics. Second, the majority of NMT research involving human stakeholders is directed towards advancing the state of MT development rather than ensuring the usefulness of NMT as a tool for professionals. Third, the review suggested overall narrow conceptualisations of translation productivity that were often based solely on measures of processing time or throughput.Fourth, it emerged that NMT investigations involving end-users are still relatively scarce. We present and discuss these findings, and make recommendations for future research on topics including the concept of productivity and the role of NMT as a professional tool.

show abstract

“…As NMT systems are sensitive to noise in parallel data (Pinnis et al, 2017a), all parallel data were filtered using the parallel data filtering methods described by Pinnis (2018). The parallel corpora filtering methods remove sentence pairs that have indications of data corruption or low parallelity (e.g., source-target length ratio, content overlap, digit mismatch, language adherence, etc.)…”

Section: Data Filteringmentioning

confidence: 99%

Tilde’s Machine Translation Systems for WMT 2018

Pinnis¹,

Rikters²,

Krišlauks³

2018

Proceedings of the Third Conference on Machine Translation: Shared Task Papers

Self Cite

View full text Add to dashboard Cite

The paper describes the development process of the Tilde's NMT systems that were submitted for the WMT 2018 shared task on news translation. We describe the data filtering and pre-processing workflows, the NMT system training architectures, and automatic evaluation results. For the WMT 2018 shared task, we submitted seven systems (both constrained and unconstrained) for English-Estonian and Estonian-English translation directions. The submitted systems were trained using Transformer models.

show abstract

Evaluation of Neural Machine Translation for Highly Inflected and Small Languages

Cited by 4 publications

References 12 publications

Tilde's Machine Translation Systems for WMT 2017

Tilde's Machine Translation Systems for WMT 2017

What has changed with neural machine translation? A critical review of human factors

Tilde’s Machine Translation Systems for WMT 2018

Contact Info

Product

Resources

About