Transliteration for Resource-Scarce Languages

Abstract

Today, parallel corpus-based systems dominate the transliteration landscape. But the resourcescarce languages do not enjoy the luxury of large parallel transliteration corpus. For these languages, rule-based transliteration is the only viable option. In this article, we show that by properly harnessing the monolingual resources in conjunction with manually created rule base, one can achieve reasonable transliteration performance. We achieve this performance by exploiting the power of Character Sequence Modeling (CSM), which requires only monolingual resources. We present the results of our rule-based system for Hindi to English, English to Hindi, and Persian to English transliteration tasks. We also perform extrinsic evaluation of transliteration systems in the context of Cross Lingual Information Retrieval. Another important contribution of our work is to explain the widely varying accuracy numbers reported in transliteration literature, in terms of the entropy of the language pairs and the datasets involved.

Leveraging Entity Linking and Related Language Projection to Improve Name Transliteration

“…In terms of transliteration unit, existing machine transliteration models can be classified into three categories, phoneme-based (Knight and Graehl, 1997;Lee and Choi, 1998;Wan and Verspoor, 1998;Jung et al, 2000;Meng et al, 2001;Oh and Choi, 2002;Virga and Khudanpur, 2003;Gao et al, 2005), grapheme-based Ekbal et al, 2006;Ganesh et al, 2008;Das et al, 2009;Chinnakotla et al, 2010;Finch and Sumita, 2010), and hybrid (Al-Onaizan and Knight, 2002;Bilac and Tanaka, 2004;Oh and Choi, 2005;Oh et al, 2006;Kim et al, 1999).…”

Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning

“…Chinnakotla et al (2010) generate transliteration candidates using manually developed character mapping rules and rerank them with a character language model. The major limitations are: (i) character transliteration probability is not learnt, so there is undue reliance on the language model to handle ambiguity, and (ii) significant manual effort for good coverage of mapping rules.…”

“…Unsupervised transliteration can be defined as: Learn a transliteration model (T X ) from the source language (F) to the target (E) language given their respective monolingual word lists, W F and W E respectively. We explore this direction in the present work, addressing shortcomings in the previous work (Ravi and Knight, 2009;Chinnakotla et al, 2010).…”

Translation approaches in Cross Language Information Retrieval

“…Parallel corpora is used to construct bilingual dictionaries. Chinnakotla et.al [21] proposed that to achieve reasonable transliteration performance when compared to that of baseline statistical systems trained using parallel corpora, combine monolingual resources with manually created rule base. For many language pairs parallel corpora is not always available.…”