A generalised alignment template formalism and its application to the inference of shallow-transfer machine translation rules from scarce bilingual corpora

Abstract

Statistical and rule-based methods are complementary approaches to machine translation (MT) that have different strengths and weaknesses. This complementarity has, over the last few years, resulted in the consolidation of a growing interest in hybrid systems that combine both data-driven and linguistic approaches. In this paper we address the situation in which the amount of bilingual resources that is available for a particular language pair is not sufficiently large to train a competitive statistical MT system, but the cost and slow development cycles of rule-based MT systems cannot be afforded either. In this context, we formalise a new method that uses scarce parallel corpora to automatically infer a set of shallowtransfer rules to be integrated into a rule-based MT system, thus avoiding the need for human experts to handcraft these rules.Our work is based on the alignment template approach to phrase-based statistical MT, but the definition of the alignment template is extended to encompass different generalisation levels. It is also greatly inspired by the work of Sánchez-Martínez and Forcada published in 2009 (Journal of Artificial Intelligence Research 34) in which alignment templates were also considered for shallow-transfer rule inference. However, our approach overcomes many relevant limitations of that work, principally those related to the inability to find the correct generalisation level for the alignment templates, and to select the subset of alignment templates that ensures an adequate segmentation of the input sentences by the rules eventually obtained. Unlike previous approaches in literature, our formalism does not require linguistic knowledge about the languages involved in the translation. Moreover, it is the first time that conflicts between rules are resolved by choosing the most appropriate ones according to a global minimisation function rather than proceeding in a pairwise greedy fashion.Experiments conducted using five different language pairs with the free/open-source rule-based MT platform Apertium show that translation quality significantly improves when compared to the method proposed by Sánchez-Martínez and Forcada (2009), and is close to that obtained using handcrafted rules. For some language pairs, our approach is even able to outperform them. Moreover, the resulting number of rules is considerably smaller, which eases human revision and maintenance.

How much hybridization does machine translation Need?

“…Tyers, Sánchez‐Martínez, and Forcada () used statistical lexical semantics to disambiguate translation inputs in the source. Sánchez‐Cartagena, Pérez‐Ortiz, and Sánchez‐Martínez () generated transfer rules (without probabilities) from a parallel corpus. These rules can then be used in combination with manually written transfer rules, if available (Costa‐jussà & Centelles, in press).…”

“…When it comes to increasing the amount of hybridization, there are some cases where adding linguistic knowledge penalizes the statistical MT system, for example, as proven by limiting translation units (Koehn et al, 2003) or reordering with linguistic criteria (Costa-jussà et al, 2007). Likewise, in the case of rule-based MT, using statistical data can be harmful, as it has been observed that in general manual transfer rules still achieve better results compared to the automatically extracted ones (Sánchez-Cartagena et al, 2015).…”

“…On the other hand, also inspired by previous techniques (Sánchez-Cartagena et al, 2015), we identified some ways of building a restricted hybrid architecture given a statistical-based MT system along with additional linguistic resources. Starting from a core statistical-based system, we first need to integrate transfer-rules and find the best way to have them coupled with the translation model.…”

The UA-Prompsit hybrid machine translation system for the 2014 Workshop on Statistical Machine Translation

“…Finally, the translation into TL is generated from the TL lexical forms: J'aime la maison de Pierre. 3.2 A rule formalism suitable for rule inference Figure 1 shows the second rule applied in the example from the previous section encoded with the formalism we have defined for rule inference (Sánchez-Cartagena et al, 2014). Each rule contains a sequence of SL word classes (depicted as the sequence of boxes at the top of the figure) and TL word classes (the sequence of boxes below them).…”

“…They are a subset of the steps followed by Sánchez-Cartagena et al (2014) to infer shallow-transfer rules to be used in Apertium from small parallel corpora.…”