Machine transliteration and transliterated text retrieval: a survey

Abstract

Users of the WWW across the globe are increasing rapidly. According to Internet live stats there are more than 3 billion Internet users worldwide today and the number of non-English native speakers is quite high there. A large proportion of these non-English speakers access the Internet in their native languages but use the Roman script to express themselves through various communication channels like messages and posts. With the advent of Web 2.0, user-generated content is increasing on the Web at a very rapid rate. A substantial proportion of this content is transliterated data. To leverage this huge information repository, there is a matching effort to process transliterated text. In this article, we survey the recent body of work in the field of transliteration. We start with a definition and discussion of the different types of transliteration followed by various deterministic and non-deterministic approaches used to tackle transliteration-related issues in machine translation and information retrieval. Finally, we study the performance of those techniques and present a comparative analysis of them.

Towards a Broad Coverage Named Entity Resource: A Data-Efficient Approach for Many Diverse Languages

“…By design, the silver data provides only a single translation for each English NE. However, multiple translations are often correct, due to the variability of morphology, transliteration, naming conventions and dialects (Prabhakar and Pal, 2018). For example, the English NE "Paul" can be aligned to "Pavel" and "Pavla".…”

GRT: Gurmukhi to Roman Transliteration System using Character Mapping and Handcrafted Rules

“…The content written on Facebook , more than 33% comments are written using phonetic text and more than 38% comments are written using code mixed phonetic text (bilingual) [4]. These text do not follow any standard spelling rules, but are based on the pronunciation of the words [5]. So, the development of phonetic dataset(s) is required for the text mining, opinion mining, information retrieval, feedback analysis, business intelligence, data analytics, etc.…”