"Answer ka type kya he?"

Abstract

Code-Mixing (CM) is defined as the embedding of linguistic units such as phrases, words, and morphemes of one language into an utterance of another language. CM is a natural phenomenon observed in many multilingual societies. It helps in speeding-up communication and allows wider variety of expression due to which it has become a popular mode of communication in social media forums like Facebook and Twitter. However, current Question Answering (QA) research and systems only support expressing a question in a single language which is an unrealistic and hard proposition especially for certain domains like health and technology. In this paper, we take the first step towards the development of a full-fledged QA system in CM language which is building a Question Classification (QC) system. The QC system analyzes the user question and infers the expected Answer Type (AType). The AType helps in locating and verifying the answer as it imposes certain type-specific constraints.We learn a basic Support Vector Machine (SVM) based QC system for English-Hindi CM questions. Due to the inherent complexities involved in processing CM language and also the unavailability of language processing resources such POS taggers, Chunkers, Parsers, we design our current system using only word-level resources such as language identification, transliteration and lexical translation. To reduce data sparsity and leverage resources available in a resourcerich language, in stead of extracting features directly from the original CM words, we translate them commonly into English and then perform featurization. We created an evaluation dataset for this task and our system achieves an accuracy of 63% and 45% in coarse-grained and fine-grained categories of the question taxonomy. The idea of translating features into English indeed helps in improving accuracy over the uni-gram baseline.

A Survey of Code-switched Speech and Language Processing

“…• A first step towards creating a code-switched QA dataset was attempted by collecting 3000 questions from a version of a TV show "Who wants to be a Millionaire?" and general knowledge questions from primary school textbooks for Hindi-English code-switching questions [55]. Out of the 3000 questions, 1000 unique questions are used in order to avoid any individual biases of language usage.…”

“…• In lieu of addressing the possibility of lexical bias from entrainment in [55], another effort was made on a larger scale to collect 5933 questions for Hindi-English, Tamil-English, Telugu-English grounded on articles and images [56].…”

Transliteration Better than Translation? Answering Code-mixed Questions over a Knowledge Base

“…CodeMixing and CodeSwitching Codemixing and code-switching has recently gathered much attention from researchers (Bhat et al, 2018;Rijhwani et al, 2017;Raghavi et al, 2015Raghavi et al, , 2017Banerjee et al, 2016;Dey and Fung, 2014;Bhat et al, 2017). CM research is mostly confined towards developing parsers and other language pipeline primitives (Bhat et al, 2018(Bhat et al, , 2017.…”

“…There has been some work in CM sentiment analysis (Joshi et al, 2016). Raghavi et al (2015) demonstrate question type classification for CM questions and Raghavi et al (2017) also demonstrate a CM factoid QA system that searches for the lexically translated CM question using Google Search on a small dataset of 100 CM questions. To the best of our knowledge, there has been no work on building an end-to-end CM QA system over a KB.…”

Detecting Multi-label emotions from code-mixed Facebook Status Updates

“…In this study, we are using eight primary emotions (joy, sadness, anger, fear, trust, disgust, surprise, and anticipation) and one secondary emotion (love) as described in Plutchik's wheel of emotions (10) . [Figure 1] There have been several early approaches for analysis of code-mixed data, mostly focusing on pre-processing (11)(12)(13)(14) , language identification (15)(16)(17)(18) , lexicon building (19) , sentiment classification (20)(21)(22) and subjectivity analysis (23) . Traditional algorithms are built to work on single-label classification problems.…”