Neural Machine Translation: Fine-Grained Evaluation of Google Translate Output for English-to-Arabic Translation

Abstract

The neural machine translation (NMT) revolution is upon us. Since 2016, an increasing number of scientific publications have examined the improvements in the quality of machine translation (MT) systems. However, much remains to be done for specific language pairs, such as Arabic and English. This raises the question whether NMT is a useful tool for translating text from English to Arabic. For this purpose, 100 English passages were obtained from different broadcasting websites and translated using NMT in Google Translate. The NMT outputs were reviewed by three professional bilingual evaluators specializing in linguistics and translation, who scored the translations based on the translation quality assessment (QA) model. First, the evaluators identified the most common errors that appeared in the translated text. Next, they evaluated adequacy and fluency of MT using a 5-point scale. Our results indicate that mistranslation is the most common type of error, followed by corruption of the overall meaning of the sentence and orthographic errors. Nevertheless, adequacy and fluency of the translated text are of acceptable quality. The results of our research can be used to improve the quality of Google NMT output.

Evaluation of google translate in rendering English COVID-19 texts into Arabic

“…The study aligns with (Z. Almahasees, 2020) and (Alkhawaja et al, 2020) that NMT proved significant progress in translating English into Arabic.…”

An Evaluation of the Accuracy of the Machine Translation Systems of Social Media Language

“…Short passages (1-10) were about 1000-1200 characters. Middle passages (11)(12)(13)(14)(15)(16)(17)(18)(19)(20) were about 2000-2400 characters. Long passages (21)(22)(23)(24)(25)(26)(27)(28)(29)(30) were about 3600-3900 characters.…”

“…Cheng [16] mentions that NMT employs a neural network that deals with input through various layers before it goes out. It uses deep learning techniques that results in quicker translation outcomes [17]. This enhancement of Google Translate is marked with both high-quality processing of translation and speed.…”

“…In this section, we explore a number of studies related to human evaluation and automatic evaluation. Alkhawaja, et al [17] say that the product of machine translation can be evaluated by human translators who have access to both the source and target languages. They evaluate a text in terms of adequacy, fluency, accuracy and cognition.…”

“…Further, their findings concur Al-khresheh"s in that the system was incapable of handling Arabic sayings and proverbs. The ungrammatical structure that Google Translate unintelligibly produces is usually related to wrong word order, mispositions of verbs in the sentences, lexicosemantic slips and probably incorrect tense concordance [17]. Inaccuracy of Google Translation output was also evident in a study conducted by Hijazi [19] who evaluated the translation of English into Arabic legal texts.…”