IIIT-H System Submission for FIRE2014 Shared Task on Transliterated Search

Bhat, Irshad Ahmad; Mujadia, Vandan; Tammewar, Aniruddha; Bhat, Riyaz Ahmad; Shrivastava, Manish

doi:10.1145/2824864.2824872

Cited by 39 publications

(17 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Edit distance technique was used for query expansion. In retrieval, applying query expansion on test queries, top 15-20 variations were selected as seed values, which were further used to generate top-20 documents [119,124].…”

Section: Transliterated Searchmentioning

confidence: 99%

Machine transliteration and transliterated text retrieval: a survey

Prabhakar

Pal

2018

Sādhanā

View full text Add to dashboard Cite

Users of the WWW across the globe are increasing rapidly. According to Internet live stats there are more than 3 billion Internet users worldwide today and the number of non-English native speakers is quite high there. A large proportion of these non-English speakers access the Internet in their native languages but use the Roman script to express themselves through various communication channels like messages and posts. With the advent of Web 2.0, user-generated content is increasing on the Web at a very rapid rate. A substantial proportion of this content is transliterated data. To leverage this huge information repository, there is a matching effort to process transliterated text. In this article, we survey the recent body of work in the field of transliteration. We start with a definition and discussion of the different types of transliteration followed by various deterministic and non-deterministic approaches used to tackle transliteration-related issues in machine translation and information retrieval. Finally, we study the performance of those techniques and present a comparative analysis of them.

show abstract

Section: Transliterated Searchmentioning

confidence: 99%

Machine transliteration and transliterated text retrieval: a survey

Prabhakar

Pal

2018

Sādhanā

View full text Add to dashboard Cite

show abstract

“…Hence, we identify the language of each word in the navigation instructions. We used an off-the-shelf system for language identification [8] which uses character ngrams as features. Due to the specificity of the domain, we also attempt to mitigate errors made by the system by labeling common words like 'road', 'bus', 'main' as English words.…”

Section: Language Identificationmentioning

confidence: 99%

“…To map the representation of native words to their corresponding phonemes used in the front end, these words are transliterated from the Romanized script to the native script. [8] modeled transliteration as a structured prediction problem using second order Hidden Markov Models. In our initial experiments using Soundex codes, we mapped these transliterated words to words from large text of monolingual script (including wiki text dump, wiki titles and web pages from relevant queries) to derive a locality name.…”

Section: Transliterationmentioning

confidence: 99%

Speech Synthesis for Mixed-Language Navigation Instructions

et al. 2017

View full text Add to dashboard Cite

Text-to-Speech (TTS) systems that can read navigation instructions are one of the most widely used speech interfaces today. Text in the navigation domain may contain named entities such as location names that are not in the language that the TTS database is recorded in. Moreover, named entities can be compound words where individual lexical items belong to different languages. These named entities may be transliterated into the script that the TTS system is trained on. This may result in incorrect pronunciation rules being used for such words. We describe experiments to extend our previous work in generating code-mixed speech to synthesize navigation instructions, with a mixed-lingual TTS system. We conduct subjective listening tests with two sets of users, one being students who are native speakers of an Indian language and very proficient in English, and the other being drivers with low English literacy, but familiarity with location names. We find that in both sets of users, there is a significant preference for our proposed system over a baseline system that synthesizes instructions in English.

show abstract

“…For language identification, we use the Language Identification tool that was developed for FIRE 2014 shared task [4] which has an accuracy of 79.2% . This system is based on a SVM trained for each language pair which makes use of character-based smoothed n-gram language models trained separately for each language.…”

Section: Question Classification Systemmentioning

confidence: 99%

"Answer ka type kya he?"

Raghavi

Chinnakotla

Shrivastava

2015

Proceedings of the 24th International Conference on World Wide Web

Self Cite

View full text Add to dashboard Cite

Code-Mixing (CM) is defined as the embedding of linguistic units such as phrases, words, and morphemes of one language into an utterance of another language. CM is a natural phenomenon observed in many multilingual societies. It helps in speeding-up communication and allows wider variety of expression due to which it has become a popular mode of communication in social media forums like Facebook and Twitter. However, current Question Answering (QA) research and systems only support expressing a question in a single language which is an unrealistic and hard proposition especially for certain domains like health and technology. In this paper, we take the first step towards the development of a full-fledged QA system in CM language which is building a Question Classification (QC) system. The QC system analyzes the user question and infers the expected Answer Type (AType). The AType helps in locating and verifying the answer as it imposes certain type-specific constraints.We learn a basic Support Vector Machine (SVM) based QC system for English-Hindi CM questions. Due to the inherent complexities involved in processing CM language and also the unavailability of language processing resources such POS taggers, Chunkers, Parsers, we design our current system using only word-level resources such as language identification, transliteration and lexical translation. To reduce data sparsity and leverage resources available in a resourcerich language, in stead of extracting features directly from the original CM words, we translate them commonly into English and then perform featurization. We created an evaluation dataset for this task and our system achieves an accuracy of 63% and 45% in coarse-grained and fine-grained categories of the question taxonomy. The idea of translating features into English indeed helps in improving accuracy over the uni-gram baseline.

show abstract

IIIT-H System Submission for FIRE2014 Shared Task on Transliterated Search

Cited by 39 publications

References 5 publications

Machine transliteration and transliterated text retrieval: a survey

Machine transliteration and transliterated text retrieval: a survey

Speech Synthesis for Mixed-Language Navigation Instructions

"Answer ka type kya he?"

Contact Info

Product

Resources

About