AdaBLEU: A Modified BLEU Score for Morphologically Rich Languages

Chauhan, Shweta; Daniel, Philemon; Mishra, A. N.; Kumar, Abhay

doi:10.1080/03772063.2021.1962745

Cited by 11 publications

(7 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, there are several fully Automatic Machine Translation Evaluation (AMTE) metrics. They can be classified into five categories [4]: lexical [31,23], character [30], semantic [18,24], syntactic [3,13,19,5], and semantic-syntactic metrics [7].…”

Section: State Of the Artmentioning

confidence: 99%

Syntactic Quality Measurement in Machine Translation with Interlinguas

Maisto,

Oliver

2023

Proceedings of the International Conference on Human-Informed Translation and Interpreting Technology 2023

View full text Add to dashboard Cite

The measurement of machine translation (MT) performances is an unsolved issue in NLP. This task can be done by a human but the time cost and the need for skilled workers to do it rise the necessity for automatic ways to measure the quality of a translation. In this work, we aim to develop a new methodology for measuring the quality of MT results from a syntactic point of view. The idea takes as a theoretical framework the work of Harris about the decomposition of sentences in elementary units called kernels. Our model parses Spanish sentences and UNL (Universal Networking Language) Graphs with a rule-based methodology and divides them into units of information.Comparing those units the model measures the quality of the translation. Our results show that decomposing the sentences in minimal syntactic units could improve the evaluation performances also without a lexical/semantic analysis.

show abstract

Section: State Of the Artmentioning

confidence: 99%

Syntactic Quality Measurement in Machine Translation with Interlinguas

Maisto,

Oliver

2023

Proceedings of the International Conference on Human-Informed Translation and Interpreting Technology 2023

View full text Add to dashboard Cite

show abstract

“…The accuracy of the module has been measured using bilingual evaluation understudy (BLEU) [23] score. It is a technique to compare machine-translated sentences to a collection of reference sentences.…”

Section: Bidirectional Translatormentioning

confidence: 99%

Inclusive bidirectional conversion system between Chittagonian and standard Bangla

Hossain

Milon²,

Sabbir

et al. 2022

Bulletin EEI

View full text Add to dashboard Cite

In recent years, the Bangla language has come out as a very prominent figure in the world of natural language processing. Many researchers have produced exemplary works on the language, but there has not been any notable work on its highly enriched dialects due to the lack of resources and the diversity and complexity in its grammatical structure. This paper has suggested a bidirectional conversion system for one of the most widely used dialects of Bangla; the Chittagonian dialect. Our method employs a bidirectional lexicon that uses binary search, word-to-word mapping, and morphological transformations. The system has achieved an accuracy rate of 95.86% for Chittagonian to standard Bangla and 93.89% in the case of standard Bangla to Chittagonian. We have also provided an efficient word suggestion module, and it has yielded satisfactory results.

show abstract

“…The BLEU metric is statistically based and is suitable for every language. Yet, it does not consider lexical relations of words (Chauhan et al, 2021), which is an important issue for highly inflective languages such as Lithuanian. Since BLEU scores have been proven to correlate less well with human evaluations in such languages, researchers and developers have proposed alternative or modified evaluation scores, e.g.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Since BLEU scores have been proven to correlate less well with human evaluations in such languages, researchers and developers have proposed alternative or modified evaluation scores, e.g. AdaBLEU, which takes into account lexical and syntactical properties of morphologically rich languages (Chauhan et al, 2021). Besides, the reliability of the metrics with low-resource languages has not been confirmed either (Kocmi et al, 2021).…”

Section: Literature Reviewmentioning

confidence: 99%

Machine Translation for Post-Editing Practices

Povilaitienė

Kasperẹ

2022

ctld

View full text Add to dashboard Cite

It has been proven in many quality-focused studies that machine translation output in some language pairs is still far from publishable (Koponen, 2016). Even so, post-editing has become a daily practice among translators and translation service providers, especially with language pairs where machine translation demonstrates good human parity. The fast development of machine translation and its quality improvement have led to a growing demand of post-editors. This study attempts to evaluate the quality of the most popular machine translation tools for the Lithuanian language in order to find the correlation between the results of automatic quality estimation (i.e., the BLUE score), human / manual evaluation of machine translation output quality following the multidimensional quality metrics (MQM) and the most common machine translation engines used by freelancers and language service providers. The conclusions are based on the findings of a survey and the automatic vs human / manual machine translation quality analysis. The findings demonstrate and support previous research that automatic machine translation quality estimation may not be taken for granted. Human / manual machine translation quality evaluation is still a better indicator whether a machine translation tool fits the purpose of translation. The study brings to the fore some insightful findings that may be beneficial for translator and post-editor trainers from the pedagogical perspective as well as for translation industry from the practical perspective.

show abstract

AdaBLEU: A Modified BLEU Score for Morphologically Rich Languages

Cited by 11 publications

References 7 publications

Syntactic Quality Measurement in Machine Translation with Interlinguas

Syntactic Quality Measurement in Machine Translation with Interlinguas

Inclusive bidirectional conversion system between Chittagonian and standard Bangla

Machine Translation for Post-Editing Practices

Contact Info

Product

Resources

About