Multi-Task Word Alignment Triangulation for Low-Resource Languages

Abstract

We present a multi-task learning approach that jointly trains three word alignment models over disjoint bitexts of three languages: source, target and pivot. Our approach builds upon model triangulation, following Wang et al., which approximates a source-target model by combining source-pivot and pivot-target models. We develop a MAP-EM algorithm that uses triangulation as a prior, and show how to extend it to a multi-task setting. On a low-resource Czech-English corpus, using French as the pivot, our multi-task learning approach more than doubles the gains in both Fand Bleu scores compared to the interpolation approach of Wang et al. Further experiments reveal that the choice of pivot language does not significantly affect performance.

Noisy Parallel Data Alignment

“…Our work is a natural extension of previous word alignment work. A robust alignment tool for lowresource languages benefits MT systems (Xiang et al, 2010a;Levinboim and Chiang, 2015;Beloucif et al, 2016a;Nagata et al, 2020), or speech recognition , especially if sentence-level alignment tools like LASER (Artetxe and Schwenk, 2019;Chaudhary et al, 2019) do not cover all languages, so one may need to fall-back to word-level alignment heuristics to inform sentence-alignment models like Hunalign (Varga et al, 2007).…”

“…As awesome-align achieves the overall highest performance, we choose to focus on awesome-align in this work. Some works involve improving word-level alignment for low-resource languages such as utilizing semantic information (Beloucif et al, 2016b;Pourdamghani et al, 2018), multi-task learning (Levinboim andChiang, 2015), and combining complementary word alignments (Xiang et al, 2010b). None of the previous work, though, to our knowledge, tackles the problem of aligning data with OCR-like noise on one or both sides.…”

Noisy Parallel Data Alignment

“…Our work is a natural extension of previous word alignment work. A robust alignment tool for lowresource languages benefits MT systems (Xiang et al, 2010a;Levinboim and Chiang, 2015;Beloucif et al, 2016a;Nagata et al, 2020), or speech recognition , especially if sentence-level alignment tools like LASER (Artetxe and Schwenk, 2019;Chaudhary et al, 2019) do not cover all languages, so one may need to fall-back to word-level alignment heuristics to inform sentence-alignment models like Hunalign (Varga et al, 2007).…”

Cross-lingual alignment transfer: a chicken-and-egg story?

“…A variant of this strategy is to view the source parameter values as priors for the target model, an idea that has been used repeatedly in the context of domain adaptation. It has notably been used for transferring parsers (Cohen and Smith, 2009;Burkett et al, 2010; and, more recently, to also transfer alignment models (Levinboim and Chiang, 2015).…”

“…In this study, we explore ways to overcome this paradox and consider techniques for transferring alignment models or annotations across language pairs, a task that has hardly been addressed in literature (see however (Wang et al, 2006;Levinboim and Chiang, 2015)). Based on a high-level typology of cross-lingual transfer methodologies ( § 2), our contribution is to formalize realistic scenarios (defined in § 3) as well as some basic methodologies for projecting knowledge about bilingual alignments crosslinguistically ( § 4).…”