Neural Machine Translation for Low-resource English-Bangla

Abstract

Islam. This open access article is distributed under a Creative Commons Attribution (CC-BY) 3.0 license.

Bangla↔English Machine Translation Using Attention-based Multi-Headed Transformer Model

“…A few parallel corpora are available for Bangla-English MT. In this study, SUPara (Al Mumin et al, 2012) dataset is used as a number of recent studies have used this corpus (Al Mumin et al, 2019aMumin et al, , 2019bHasan et al, 2019aHasan et al, , 2019b. The dataset contains 70861, 500 and 500 parallel sentences for training, validation and test sets, respectively.…”

shu-torjoma: An English↔Bangla Statistical Machine Translation System

“…In our experiment, we used Shahjalal University parallel (SUPara) (Mumin et al, 201; 2018b) corpus and GolbalVoices (Tiedemann, 2012) corpus from OPUS (Tiedemann, 2012) as a training dataset. SUPara (Mumin et al, 2012;2018b) is a balanced corpus consists of texts from different genres like literature, journalistic texts, instructive texts, administrative texts, and texts treating external communication, which are collected from various printed and online media. GolbalVoices (Tiedemann, 2012) corpus consists of only news texts collected form GlobalVoices website iv .…”

“…These two datasets were developed with a vision of using them as a benchmark in English-Bangla MT research. The texts of these two datasets were well-chosen from balanced SUPara (Mumin et al, 2012;2018b) corpus, thus these two datasets are also balanced in genre. In addition, to make these datasets representative in length we selected the texts from 10 subsets of different lengths: 1 to 5 words, 6 to 10 and so forth up to 40 to 45 and finally longer than 45 words.…”

“…Our core PBMT system is implemented using the Moses (Koehn et al, 2007) SMT toolkit. We trained our system on English-Bangla parallel training dataset which is a combination of SUPara (Mumin et al, 2012;2018b) and GlobalVoices (Tiedemann, 2012) corpus. We extracted symmetrized word alignments from this training dataset using GIZA++ (Och and Ney, 2003) and grow-diag-final-and heuristic.…”

“…En→Bn. For En→Bn translation task, we trained our system on English-Bangla parallel training dataset which is a combination of SUPara (Mumin et al, 2012;2018b) and GlobalVoices (Tiedemann, 2012) corpus and on Bangla monolingual dataset, SUMono (Mumin et al, 2014). Then, we tuned our system using Minimum Error Rate Training (MERT) (Och, 2003) on the Bangla side of the development dataset, SUParadev2018 (Mumin et al, 2018a), so as to maximize the BLEU (Papineni et al, 2002) score.…”