Evaluating Translation Tools: Google Translate, Bing Translator, and Bing AI on Arabic Colloquialisms

Abstract

This study examines the advancements in AI-driven machine translation, specifically focusing on the accurate translation of Arabic colloquial expressions. It aims to assess the progress made by Large Language Models, such as Bing AI Chat, compared to traditional machine translation systems. By focusing on colloquial expressions, this research aims to shed light on the challenges and opportunities for improvement in machine translation systems, particularly when dealing with the complexities of translating informal Arabic utterances. Building upon At-tall’s 2019 thesis, which compared Google Translate and human translators, the study employs the same Arabic sentences as a test dataset, allowing for a direct comparison between 2019 translations and those produced by current machine translation tools. The findings indicate limited improvement in Google Translate since 2019, with Bing Translator exhibiting a similar level of translation accuracy. In contrast, Bing AI Chat consistently outperformed the other systems, showcasing the potential of Large Language Model machine translation. Notably, Bing AI Chat provided interpretations and valuable comments on the tested Arabic phrases, demonstrating a deeper understanding of the intended meaning. This study contributes significantly to the field of machine translation by providing evidence of the potential of Large Language Model systems in producing more accurate Arabic-English translations. It emphasizes the advantage of Large Language Models in dealing with non-standard Arabic expressions, encouraging further exploration of Large Language Model-powered approaches in machine translation. The findings offer a promising pathway towards achieving more accurate and expressive translations across diverse languages and cultures.