Report of NEWS 2009 machine transliteration shared task

Abstract

This report documents the details of the Machine Transliteration Shared Task conducted as a part of the Named Entities Workshop (NEWS), an ACL-IJCNLP 2009 workshop. The shared task features machine transliteration of proper names from English to a set of languages. This shared task has witnessed enthusiastic participation of 31 teams from all over the world, with diversity of participation for a given system and wide coverage for a given language pair (more than a dozen participants per language pair). Diverse transliteration methodologies are represented adequately in the shared task for a given language pair, thus underscoring the fact that the workshop may truly indicate the state of the art in machine transliteration in these language pairs. We measure and report 6 performance metrics on the submitted results. We believe that the shared task has successfully achieved the following objectives: (i) bringing together the community of researchers in the area of Machine Transliteration to focus on various research avenues, (ii) Calibrating systems on common corpora, using common metrics, thus creating a reasonable baseline for the state-of-the-art of transliteration systems, and (iii) providing a quantitative basis for meaningful comparison and analysis between various algorithmic approaches used in machine transliteration. We believe that the results of this shared task would uncover a host of interesting research problems, giving impetus to research in this significant research area. English to Hindi English to Tamil English to Kannada English to Russian English to ChineseEnglish to Korean English to Japanese Katakana

A noisy channel model for grapheme-based machine transliteration

“…Table 3. E2C tuning performance (back) Table 2 shows that forward transliteration performance gets improved with the increase of tuning set size, so we use the whole development set as the tuning set to tune the final system and the final official results from the shared task report (Li et al, 2009b) Experiments show that forward transliteration has better performance than back transliteration. One reason may be that on average English name is longer than Chinese name, thus need more data to train a good character level language model.…”

Noise-aware Character Alignment for Extracting Transliteration Fragments

“…We also used two additional metrics for character-wise evaluation: F-score and BLEU c . F-score is a character-wise F-measure-like score (Li, Kumaran, Zhang, and Pervouchine 2010). BLEU c is BLEU (Papineni, Roukos, Ward, and Zhu 2002) at the character level with n=4.…”

Hindi to English Machine Transliteration of Named Entities using Conditional Random Fields

“…ACC = 1 means that all top candidates are correct transliterations i.e. they match one of the references, and ACC = 0 means that none of the top candidates are correct [46].…”