Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine

Abstract

This report provides a preliminary evaluation of ChatGPT for machine translation, including translation prompt, multilingual translation, and translation robustness. We adopt the prompts advised by ChatGPT to trigger its translation ability and find that the candidate prompts generally work well and show minor performance differences. By evaluating on a number of benchmark test sets 1 , we find that ChatGPT performs competitively with commercial translation products (e.g., Google Translate) on high-resource European languages but lags behind significantly on lowresource or distant languages. For distant languages, we explore an interesting strategy named pivot prompting that asks ChatGPT to translate the source sentence into a highresource pivot language before into the target language, which improves the translation performance significantly. As for the translation robustness, ChatGPT does not perform as well as the commercial systems on biomedical abstracts or Reddit comments but is potentially a good translator for spoken language.

Opportunities and Challenges of ChatGPT in Academia: A Conceptual Analysis

“…(c) Personal Skill Development: • Improve language skills: ChatGPT can play a significant role to improve individual language skills. This model is able to perform machine translation to overcome language barriers and make information more accessible 20 . Again, this model can improve users' language proficiency by answering questions and providing practice (speaking and writing) sessions in real time 19 .…”

“…• Provide language translation: ChatGPT can translate text from one language to another, providing real-time translation services and improving communication between individuals using different languages 20 . Moreover, ChatGPT can aid human translators by offering alternate phrasing, defining uncommon terms, and enhancing the accuracy of the translation process.…”

ChatGPT a breakthrough in science and education: Can it fail a test?

“…The results are given in parenthesis at the end of each paragraph below: The instruction given to ChatGPT referred to a very broad area making it relatively easy for ChatGPT to generate an adequate and relatively straightforward answer, which could indeed pass for an editorial, but certainly for the start of an introduction section of a student paper. The system may have a more challenging time with a narrower topic in the sports field, as shown by linguistics, mathematics, law, and medicine studies (Choi et., al, 2023;Frieder et al, 2023;Jeblick et al, 2022;Jiao et al, 2023). But each academic area is unique; some are philosophical while others are more factual, such as those in natural sciences, and ChatGPT must adapt to handling this range of information.…”

“…On the negative side, a recent study concluded that ChatGPT might not perform as well as commercial systems for biomedical abstracts or Reddit comments regarding translation accuracy. Still, translating spoken language may be of good service (Jiao et al, 2023). Additionally, apart from linguistics, ChatGPT was test-driven in mathematics, and the conclusion was that the system's math ability was lower than that of an average graduate student in mathematics (Frieder et al, 2023).…”

Exploring GPT-3 Model's Capability in Passing the Sally-Anne Test A Preliminary Study in Two Languages

“…Large language models, including ChatGPT, have received both recognition and criticism for their capacity to produce human-like text (Jiao et al, 2023;Pavlik, 2023;OpenAI, 2022). While these models are trained on vast amounts of data, they have been criticized for their lack of understanding of the language they generate (Binz & Schulz, 2023).…”