Code-Mixed Question Answering Challenge: Crowd-sourcing Data and Techniques

Chandu, Khyathi Raghavi; Loginova, Ekaterina; Gupta, Vishal; Genabith, Josef van; Neumann, Günter; Chinnakotla, Manoj Kumar; Nyberg, Eric; Black, Alan W.

doi:10.18653/v1/w18-3204

Cited by 27 publications

(16 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These include tasks on transliterated search, (Roy et al, 2013;Choudhury et al, 2014) code-mixed entity extraction (Rao and Devi, 2016) and mixed script information retrieval (Sequiera et al, 2015;Banerjee et al, 2016). Other notable shared tasks include the Tool Contest on POS Tagging for Code-Mixed Indian Social Media at ICON 2016 , Sentiment Analysis for Indian Languages (Code-Mixed) at ICON 2017(Patra et al, 2018 and the Code-Mixed Question Answering Challenge (Chandu et al, 2018a).…”

Section: Relation To Prior Workmentioning

confidence: 99%

“…Question Answering is the task of answering a question based on the given context or world knowledge. We choose the dataset provided by (Chandu et al, 2018a) which contains two types of questions for En-Hi, one with context (185 article based questions) and one containing image based questions (774 questions). For the image based questions we use the DrQA -Document Retriever module 2 to extract the most relevant context from Wikipedia.…”

Section: Question Answering (Qa)mentioning

confidence: 99%

See 1 more Smart Citation

GLUECoS: An Evaluation Benchmark for Code-Switched NLP

Khanuja¹,

Dandapat²,

Srinivasan³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Code-switching is the use of more than one language in the same conversation or utterance. Recently, multilingual contextual embedding models, trained on multiple monolingual corpora, have shown promising results on cross-lingual and multilingual tasks. We present an evaluation benchmark, GLUECoS, for code-switched languages, that spans several NLP tasks in English-Hindi and English-Spanish. Specifically, our evaluation benchmark includes Language Identification from text, POS tagging, Named Entity Recognition, Sentiment Analysis, Question Answering and a new task for code-switching, Natural Language Inference. We present results on all these tasks using cross-lingual word embedding models and multilingual models. In addition, we fine-tune multilingual models on artificially generated code-switched data. Although multilingual models perform significantly better than cross-lingual models, our results show that in most tasks, across both language pairs, multilingual models fine-tuned on code-switched data perform best, showing that multilingual models can be further optimized for code-switching tasks.

show abstract

Section: Relation To Prior Workmentioning

confidence: 99%

Section: Question Answering (Qa)mentioning

confidence: 99%

GLUECoS: An Evaluation Benchmark for Code-Switched NLP

Khanuja¹,

Dandapat²,

Srinivasan³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…In attempts to progress the field of code-mixed data, several code-switching workshops (Diab et al, 2014Aguilar et al, 2018b) have been organized in notable conferences. Most of the workshops include shared tasks on various of the lan-guage understanding tasks like language identification (Solorio et al, 2014;Molina et al, 2016), NER (Aguilar et al, 2018a;Rao and Devi, 2016), IR (Roy et al, 2013;Banerjee et al, 2018), PoS tagging (Jamatia et al, 2016), sentiment analysis (Patra et al, 2018;Patwa et al, 2020), and question answering (Chandu et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

CoMeT: Towards Code-Mixed Translation Using Parallel Monolingual Sentences

Gautam¹,

Kodali²,

Gupta³

et al. 2021

Proceedings of the Fifth Workshop on Computational Approaches to Linguistic Code-Switching

View full text Add to dashboard Cite

Code-mixed languages are very popular in multilingual societies around the world, yet the resources lag behind to enable robust systems on such languages. A major contributing factor is the informal nature of these languages which makes it difficult to collect codemixed data. In this paper, we propose our system for Task 1 of CACLS 2021 1 to generate a machine translation system for English to Hinglish in a supervised setting. Translating in the given direction can help expand the set of resources for several tasks by translating valuable datasets from high resource languages. We propose to use mBART, a pre-trained multilingual sequence-to-sequence model, and fully utilize the pre-training of the model by transliterating the roman Hindi words in the code-mixed sentences to Devanagri script. We evaluate how expanding the input by concatenating Hindi translations of the English sentences improves mBART's performance. Our system gives a BLEU score of 12.22 on test set. Further, we perform a detailed error analysis of our proposed systems and explore the limitations of the provided dataset and metrics.

show abstract

“…On many occasions, they use code-switching or linguistic switching (3) . The concept of using multiple languages in a single sentence is very popular among the multilingual community (4) and to perform the sentiment analysis of these statements for more accuracy, the sentence is translated into one language and then polarity is calculated. 2.…”

Section: Introductionmentioning

confidence: 99%

Annotated corpus creation for sentiment analysis in code-mixed Hindi-English (Hinglish) social network data

Garg¹,

Sharma²

2020

IJST

View full text Add to dashboard Cite

Background: Evaluating the sentiments of tweets, blogs, comments and posts have become a crucial part of many applications. Sentiment analysis of social network data is very helpful for decision-making in application areas like movie reviews, product feedback and impact of the speech of a politician etc. The users often comment in their native languages or in slang languages or more often they use abbreviations and do not even stick to grammatical rules of the language. The bilingual and multilingual community often mixes two or more than two languages in their comments. Unavailability of annotated code-mixed data for native language adds to the difficulty in performing sentiment analysis. Objectives The motive of this article is to present the process of creating annotated corpus for code mixed social media text in Hindi, English and Hinglish which is collected from twitter. Ambiguous meaning words and inconsistent spellings of both the languages have also been included in the study to provide wide spread canvas. Method: This study will provide significant elements that should be considered while developing the annotated corpus of Hindi, Hinglish & English dataset. The annotation is calculated on the basis of polarity of words in three categories as positive, negative and neutral. There are words which have mixed feeling i.e. these words have positive as well as negative sentiments. To consider these words, inner agreement among the polarities has been considered. The words used for sarcasm or slangs have also been taken into account. The study has included ambiguous meaning and inconsistent spelling words of both languages as well. Findings: The proposed work provides a standard annotated corpus for code switched social media text in Hindi-English (Hinglish). The process of developing the corpus and calculating the polarity has been shown. It is found that if one considers the code-mixed text, the accuracy can be enhanced. Application: The proposed corpus can be utilized in the area of market analysis, customer behavior, polling analysis, brand monitoring, etc. The corpus serves as dataset which can further be extended according to the problem definition.

show abstract

Code-Mixed Question Answering Challenge: Crowd-sourcing Data and Techniques

Cited by 27 publications

References 33 publications

GLUECoS: An Evaluation Benchmark for Code-Switched NLP

GLUECoS: An Evaluation Benchmark for Code-Switched NLP

CoMeT: Towards Code-Mixed Translation Using Parallel Monolingual Sentences

Annotated corpus creation for sentiment analysis in code-mixed Hindi-English (Hinglish) social network data

Contact Info

Product

Resources

About