Plan Optimization to Bilingual Dictionary Induction for Low-resource Language Families

Nasution, Arbi Haza; Murakami, Yohei; Ishida, Toru

doi:10.1145/3448215

Cited by 7 publications

(5 citation statements)

References 23 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to achieve even better results, many BLI methods also apply a self-learning loop where training dictionaries are iteratively (and gradually) refined, and improved mappings are then learned in each iteration (Artetxe et al, 2018;Karan et al, 2020). However, there is still ample room for improvement, especially for lower-resource languages and dissimilar language pairs Nasution et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

Improving Word Translation via Two-Stage Contrastive Learning

Li¹,

Liu²,

Collier³

et al. 2022

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

Word translation or bilingual lexicon induction (BLI) is a key cross-lingual task, aiming to bridge the lexical gap between different languages. In this work, we propose a robust and effective two-stage contrastive learning framework for the BLI task. At Stage C1, we propose to refine standard cross-lingual linear maps between static word embeddings (WEs) via a contrastive learning objective; we also show how to integrate it into the self-learning procedure for even more refined cross-lingual maps. In Stage C2, we conduct BLI-oriented contrastive fine-tuning of mBERT, unlocking its word translation capability. We also show that static WEs induced from the 'C2-tuned' mBERT complement static WEs from Stage C1. Comprehensive experiments on standard BLI datasets for diverse languages and different experimental setups demonstrate substantial gains achieved by our framework. While the BLI method from Stage C1 already yields substantial gains over all state-of-the-art BLI methods in our comparison, even stronger improvements are met with the full two-stage framework: e.g., we report gains for 112/112 BLI setups, spanning 28 language pairs.

show abstract

Section: Introductionmentioning

confidence: 99%

Improving Word Translation via Two-Stage Contrastive Learning

Li¹,

Liu²,

Collier³

et al. 2022

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

show abstract

“…However, implementing the constraint-based approach on a large scale to create multiple bilingual dictionaries is still challenging, particularly in determining the constraint-based approach's execution order to reduce the total costs. Plan optimization using the Markov decision process is essential when composing the order of creation for bilingual dictionaries, considering the methods and their costs [6,9].…”

Section: Bilingual Lexicon Inductionmentioning

confidence: 99%

“…Indonesian ethnic languages are low-resource languages with a limited amount of language resources, such as bilingual dictionaries. We chose Minangkabau, Malay, Palembang, Javanese, and Sundanese as the languages to implement the proposed method in this study due to the availability of the bilingual dictionaries obtained from the results of our previous study [6]. Moreover, the Indonesian and Minangkabau languages have significant lexical similarities; thus, we presume they have several phonetic transformation rules, from Indonesian to Minangkabau and vice versa.…”

Section: Introductionmentioning

confidence: 99%

Neural Network-Based Bilingual Lexicon Induction for Indonesian Ethnic Languages

2023

Self Cite

View full text Add to dashboard Cite

Indonesia has a variety of ethnic languages, most of which belong to the same language family: the Austronesian languages. Due to the shared language family, words in Indonesian ethnic languages are very similar. However, previous research suggests that these Indonesian ethnic languages are endangered. Thus, to prevent that, we propose the creation of a bilingual dictionary between ethnic languages, using a neural network approach to extract transformation rules, employing character-level embedding and the Bi-LSTM method in a sequence-to-sequence model. The model has an encoder and decoder. The encoder reads the input sequence character by character, generates context, and then extracts a summary of the input. The decoder produces an output sequence wherein each character at each timestep, as well as the subsequent character output, are influenced by the previous character. The first experiment focuses on Indonesian and Minangkabau languages with 10,277 word pairs. To evaluate the model’s performance, five-fold cross-validation was used. The character-level seq2seq method (Bi-LSTM as an encoder and LSTM as a decoder) with an average precision of 83.92% outperformed the SentencePiece byte pair encoding (vocab size of 33) with an average precision of 79.56%. Furthermore, to evaluate the performance of the neural network model in finding the pattern, a rule-based approach was conducted as the baseline. The neural network approach obtained 542 more correct translations compared to the baseline. We implemented the best setting (character-level embedding with Bi-LSTM as the encoder and LSTM as the decoder) for four other Indonesian ethnic languages: Malay, Palembang, Javanese, and Sundanese. These have half the size of input dictionaries. The average precision scores for these languages are 65.08%, 62.52%, 59.69%, and 58.46%, respectively. This shows that the neural network approach can identify transformation patterns of the Indonesian language to closely related languages (such as Malay and Palembang) better than distantly related languages (such as Javanese and Sundanese).

show abstract

“…As an example, Banjarese is similar to Malay [39] and has a 73% lexical similarity with Indonesian [12], both languages included in mC4. Minangkabau and Indonesian also have commonalities in their vocabulary and syntax [37].…”

Section: Indobart V2mentioning

confidence: 99%

Many-to-Many Multilingual Translation Model for Languages of Indonesia

Wongso,

Joyoadikusumo,

Buana

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Indonesia is home to over 700 languages and most people speak their respective regional languages aside from the lingua franca. In this paper, we focus on the task of multilingual machine translation for 45 regional Indonesian languages and introduced Indo-T5 which leveraged the mT5 sequence-to-sequence language model as a baseline. Performances of bilingual and multilingual finetuning methods were also compared, in which we found that our models have outperformed current state-of-the-art translation models. We also investigate the use of religious texts from the Bible as an intermediate mid-resource translation domain for low-resource translation domain specialization. Our findings suggest that this two-step fine-tuning approach is highly effective in improving the quality of translations for low-resource text domains. Our results show an increase in SacreBLEU scores when evaluated on the low-resource NusaX dataset. We release our translation models for other researchers to leverage.

show abstract

Plan Optimization to Bilingual Dictionary Induction for Low-resource Language Families

Cited by 7 publications

References 23 publications

Improving Word Translation via Two-Stage Contrastive Learning

Improving Word Translation via Two-Stage Contrastive Learning

Neural Network-Based Bilingual Lexicon Induction for Indonesian Ethnic Languages

Many-to-Many Multilingual Translation Model for Languages of Indonesia

Contact Info

Product

Resources

About