Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages

Abstract

Research in NLP lacks geographic diversity, and the question of how NLP can be scaled to low-resourced languages has not yet been adequately solved. "Lowresourced"-ness is a complex problem going beyond data availability and reflects systemic problems in society. * ∀ to represent the whole Masakhane community.As MT researchers cannot solve the problem of low-resourcedness alone, we propose participatory research as a means to involve all necessary agents required in the MT development process. We demonstrate the feasibility and scalability of participatory research with a case study on MT for African languages. Its implementation leads to a collection of novel translation datasets, MT benchmarks for over 30 languages, with human evaluations for a third of them, and enables participants without formal training to make a unique scientific contribution. Benchmarks, models, data, code, and evaluation results are released at https://github. com/masakhane-io/masakhane-mt.

English2Gbe: A multilingual machine translation model for {Fon/Ewe}Gbe

“…To the best of our knowledge, we are the first to suggest an NMT model to translate from English to Ewe. Our bilingual NMT model from English to Fon gives a BLEU score about 11 points greater than the current state-of-the-art (Nekoto et al, 2020). Moreover, we showed that the multilingual model performs better than bilingual models, making it the new state-of-the-art.…”

“…This paper introduces English2Gbe, a multilingual NMT model capable of translating from English to Ewe or Fon. Using the BLEU, CHRF, and TER scores computed with the Sacrebleu (Post, 2018) package for reproducibility, we show that English2Gbe outperforms bilingual models (English to Ewe and English to Fon) and gives state-of-the-art results on the JW300 benchmark for Fon established by Nekoto et al (2020). We hope this work will contribute to the massive adoption of Multilingual models inside the community.…”

“…The African continent is prosperous in more than 2,000 languages. However, most of them are low-resourced (Nekoto et al, 2020). Machine translation (MT) then emerged as a compelling technology to revive and strengthen African languages.…”

“…For instance, most African languages are excluded as they are often low and even very low-resourced. The work by Nekoto et al (2020) has been a significant step toward accounting for African languages. The Masakhane community provided an array of new datasets and more than 30 NMT benchmarks.…”

“…We trained two bilingual models, English to Ewe, English to Fon, and one multilingual model English to Fon/Ewe. We used the JW300 dataset (Agic and Vulic, 2020) and the benchmark proposed by Masakhane (Nekoto et al, 2020) to evaluate our models.…”