Quality and Machine Translation: An Evaluation of Online Machine Translation of English into Arabic Texts

Abstract

This study compares the translation outputs of an English into Arabic text using the three machine translators of Google Translate, Microsoft Bing, and Ginger. To carry this evaluation of the machine translation (MT) outputs, an English text and its Arabic counterpart were selected from the UN records. The English source text was segmented into 84 semantic chunks. Depending on the Arabic counterpart model text, each chunk was rated as "correct or incorrect" at the two levels of the translation attributes: fidelity and intelligibility. To perform the quantitative description of the evaluation process, the numbers of fidelity and intelligibility errors and their percentages were calculated. Results of this evaluation process revealed that none of the three translated versions of the source text was perfectly translated. Although the translation of Microsoft Bing was rated the best, Google's translation was found the least accurate due to the high percentage of fidelity and intelligibility errors detected in its translation output. However, the quality of Ginger's translation was found slightly less accurate than that of Microsoft Bing, but remarkably better than Google's translation. The findings of this study imply that these MT applications can be implemented to perform English into Arabic translation to get the broad gist of a source text, but a deep and thorough post-editing process looks essential for a full and accurate understanding of an English into Arabic MT output. The study recommends that more studies are encouraged to continue to assess the quality of MT that will further highlight its weaknesses and the strategies that should be adopted to overcome them.

An Evaluation of the Accuracy of the Machine Translation Systems of Social Media Language

“…Results referred to two types: omission and bad word choice. Ali [34] revealed that three MT systems, namely, Google Translate, Microsoft Bing, and Ginger, performed insufficiently. Google translate came last in terms of accuracy.…”

Systematic Review of English/Arabic Machine Translation Postediting: Implications for AI Application in Translation Research and Pedagogy

“…Ref. [65] reported that MB was the best engine, while [26] concluded that Reverso Context has a similar performance to GT. Ref.…”

“…However, the literature has also shown variation and discrepancies in evaluating Arabic MT engines. For instance, some studies show that GT was the least accurate in terms of intelligibility [65] or in dealing with legal discourse. Ref.…”

Evaluating Translation Tools: Google Translate, Bing Translator, and Bing AI on Arabic Colloquialisms

“…Machine Translation (MT) systems have made significant progress over the years, as evidenced by numerous research studies assessing their performance (Al-Jarf, 2023;Hutchins, 2001;Hutchin & Somers, 1992;Okpor, 2014). However, when it comes to translating Arabic everyday expressions, the results remain mixed (Ali, 2020;Almahasees, 2021;Ameur, Mezaine, & Guessoum, 2020;At-tall, 2019;Banimelhem & Amayreh, 2023;Harrat, Meftouh & Smaili, 2019;Zakraoui, Saleh, Al-Maadeed & Alja'am, 2021). Most significantly, At-tall (2019) conducted a thesis comparing the performance of Google Translator and human translators in rendering colloquial Arabic expressions from the late Prime Minister Wasfi At-Tall's speeches into English.…”

“…These systems employ statistical machine translation techniques and utilize vast amounts of parallel bilingual and multilingual data for training their models (Koehn, 2010;Almahasees, 2021). Despite their widespread use, both GT and BT have faced criticism for their limitations in accurately comprehending the hidden meanings of language, particularly when it comes to translating Arabic idiomatic expressions and complex linguistic structures (Aldawsari, 2023;Ali, 2020;Almahasees, 2021;Ameur, Mezaine, & Guessoum, 2020;At-tall, 2019;Harrat, Meftouh & Smaili, 2019;Zakraoui, Saleh, Al-Maadeed & Alja'am, 2021). For instance, a study conducted by Aldawsari (2023) focused on the translations of two MT systems, Google Translate and SYSTRAN, and concluded that both systems "struggled" in accurately translating complex Arabic linguistic features such as homonyms, heteronyms, and polysemes (p. 27).…”