Adaptive Machine Translation with Large Language Models

Abstract

Consistency is a key requirement of highquality translation. It is especially important to adhere to pre-approved terminology and corrected translations in domain-specific projects.Machine translation (MT) has achieved significant progress in the area of domain adaptation. However, real-time adaptation remains challenging. Large-scale language models (LLMs) have recently shown interesting capabilities of in-context learning, where they learn to replicate certain input-output text generation patterns, without further fine-tuning. By feeding an LLM with a prompt that consists of a list of translation pairs, it can then simulate the domain and style characteristics at inference time. This work aims to investigate how we can utilize in-context learning to improve real-time adaptive MT. Our extensive experiments show promising results at translation time. For example, GPT-3.5 can adapt to a set of in-domain sentence pairs and/or terminology while translating a new sentence. We observe that the translation quality with few-shot in-context learning can surpass that of strong encoder-decoder MT systems, especially for high-resource languages. Moreover, we investigate whether we can combine MT from strong encoder-decoder models with fuzzy matches, which can further improve the translation, especially for less supported languages. We conduct our experiments across five diverse languages, namely English-to-Arabic (EN-AR), Englishto-Chinese (EN-ZH), English-to-French (EN-FR), English-to-Kinyarwanda (EN-RW), and English-to-Spanish (EN-ES) language pairs.

A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4

“…In all the above discussed research works, the performances of GLLMs are just satisfactory but not on par or beyond the performances of commercial machine translation systems. Some of the research works [198]- [201], [204], [205], [207], [208] showed that it is possible to outperform commercial machine translation systems using GLLMs. For example, Jiao et al [198] investigated the translation capabilities of GLLMs like ChatGPT and GPT-4 and compared the performance with commercial systems like Google Translate, DeepL Translate and Tencent TranSmart.…”

Exploring Human-Like Translation Strategy with Large Language Models

“…These capabilities have led to LLMs exhibiting a certain degree of human-level intelligence, particularly in the areas of language understanding and generation Bubeck et al, 2023;Wu et al, 2023;Moghaddam and Honey, 2023). Among numerous tasks, translation has emerged as a prominent area where LLMs have shown impressive capacity and competence (Jiao et al, 2023b;Agrawal et al, 2023;Zhang et al, 2023a;Vilar et al, 2022;Moslem et al, 2023;Pilault et al, 2023;Hendy et al, 2023;Zhu et al, 2023b;Jiao et al, 2023a;Karpinska and Iyyer, 2023;Peng et al, 2023;Bawden and Yvon, 2023;. This progress above harkens back to the long-term aspirations and dreams of earlier machine translation research in the 1960s (Bar-Hillel, 1960;Macklovitch, 1995): Can LLMs employ a translation process similar to human translators?…”

Atlas of dynamic spectra of fast radio burst FRB 20201124A

“…Large-scale language models (LLMs) (Brown et al, 2020;Smith et al, 2022;Du et al, 2022;Rae et al, 2021;Thoppilan et al, 2022;Hoffmann et al, 2022;Chowdhery et al, 2022;Touvron et al, 2023) have shown an impressive ability for in-context learning: with only a few task-specific examples as demonstrations, LLMs are able to generate results for a new test input. Under the framework of in-context learning, LLMs have achieved promising results in a variety of NLP tasks, include machine translation (MT) (Vilar et al, 2022;Vidal et al, 2022;Moslem et al, 2023), question answering (QA) (Robinson et al, 2022;Lazaridou et al, 2022) and named entity extraction (NEE) (Chowdhery et al, 2022;Brown et al, 2020).…”