Unraveling the English-Bengali Code-Mixing Phenomenon

Chanda, Arunavha; Das, Dipankar; Mazumdar, Chandan

doi:10.18653/v1/w16-5810

Cited by 26 publications

(14 citation statements)

References 6 publications

(6 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Transliteration can be considered as a special form of code-mixing where the phonetic transformations of the words from a source language to a target language is performed. The presence of code-mixing and transliterated Bengali (i.e., Bengali text using the Latin alphabet) is a common phenomenon in Bengali, as shown by the previous studies (Barman et al, 2014;Chanda et al, 2016).…”

Section: Introductionmentioning

confidence: 75%

Evaluating the Consistency of Word Embeddings from Small Data

Bloem

Fokkens

Herbelot

2019

Proceedings - Natural Language Processing in a Deep Learning World

View full text Add to dashboard Cite

In this work, we address the evaluation of distributional semantic models trained on smaller, domain-specific texts, particularly philosophical text. Specifically, we inspect the behaviour of models using a pretrained background space in learning. We propose a measure of consistency which can be used as an evaluation metric when no in-domain gold-standard data is available. This measure simply computes the ability of a model to learn similar embeddings from different parts of some homogeneous data. We show that in spite of being a simple evaluation, consistency actually depends on various combinations of factors, including the nature of the data itself, the model used to train the semantic space, and the frequency of the learned terms, both in the background space and in the in-domain data of interest.

show abstract

Section: Introductionmentioning

confidence: 75%

Evaluating the Consistency of Word Embeddings from Small Data

Bloem

Fokkens

Herbelot

2019

Proceedings - Natural Language Processing in a Deep Learning World

View full text Add to dashboard Cite

show abstract

“…It is required to take the surrounding words into consideration in order to get a sense and context information in identifying the word (A. Chanda,et al,,2016)…”

Section: Ambiguous Wordsmentioning

confidence: 99%

Sentiment Analysis of Code-Mixed Text: A Review

et.al¹

2021

TURCOMAT

View full text Add to dashboard Cite

In recent times, sentiment analysis has become one of the most active research and progressively popular areas in information retrieval and text mining. To date, sentiment analysis has been applied in various domains such as product, movie, sport and political reviews. Most of the previous work in this field has focused on analyzing only a single language, especially English. However, with the need of globalization and the increasing number of the Internet used worldwide; it is common to see the post written in multiple languages. Moreover, in an unstructured content like Twitter posts, people tend to mix languages in one sentence, which make sentiment analysis process even harder and more challenging. This paper reviews the state-of-the-art of sentiment analysis for code-mixed, which includes the detail discussions of each focus area, qualitative comparison and limitations of current approaches. This paper also highlights challenges along this line of research and suggests several recommendations for future works that should be explored.

show abstract

“…We see our work as a part of this continuing trend and as an important resource contribution to analyze Indian social media. Romanized Indian Languages: In the context of processing Indian languages expressed on the web, challenges posed by the use of Roman script instead of the native script have been reported in several recent studies in the context of code-mixed English-Bengali (Chanda et al, 2016), and English-Hindi (Kumar et al, 2018) text. While addressing word level language identification, reported that 90% of posts in Indian languages on Facebook are expressed in Roman script.…”

Section: Related Workmentioning

confidence: 99%

Annotation Efficient Language Identification from Weak Labels

Palakodety¹,

KhudaBukhsh²

2020

Proceedings of the Sixth Workshop on Noisy User-Generated Text (W-Nut 2020)

View full text Add to dashboard Cite

India is home to several languages with more than 30m speakers. These languages exhibit significant presence on social media platforms. However, several of these widely-used languages are under-addressed by current Natural Language Processing (NLP) models and resources. User generated social media content in these languages is also typically authored in the Roman script as opposed to the traditional native script further contributing to resource scarcity. In this paper, we leverage a minimally supervised NLP technique to obtain weak language labels from a large-scale Indian social media corpus leading to a robust and annotation-efficient languageidentification technique spanning nine Romanized Indian languages. In fast-spreading pandemic situations such as the current COVID-19 situation, information processing objectives might be heavily tilted towards under-served languages in densely populated regions. We release our models to facilitate downstream analyses in these low-resource languages 1 . Experiments across multiple social media corpora demonstrate the model's robustness and provide several interesting insights on Indian language usage patterns on social media. We release an annotated data set of 1,000 comments in ten Romanized languages as a social media evaluation benchmark 1 .

show abstract

Unraveling the English-Bengali Code-Mixing Phenomenon

Cited by 26 publications

References 6 publications

Evaluating the Consistency of Word Embeddings from Small Data

Evaluating the Consistency of Word Embeddings from Small Data

Sentiment Analysis of Code-Mixed Text: A Review

Annotation Efficient Language Identification from Weak Labels

Contact Info

Product

Resources

About