Understanding Language Preference for Expression of Opinion and
            Sentiment: What do Hindi-English Speakers do on Twitter?

Rudra, Koustav; Rijhwani, Shruti; Begum, Rafiya; Bali, Kalika; Choudhury, Monojit; Ganguly, Niloy

doi:10.18653/v1/d16-1121

Cited by 57 publications

(52 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Language of social media users: We assumed that if a user is predominantly using Hindi words in a tweet then the chances of him/her being a native Hindi speaker should be high, since, while the number of English native speakers in India is 0.02%, the number of Hindi native speakers is 41.03% 4 . This assumption has also been made in earlier studies (Rudra et al, 2016). Note that even if a user is not a native Hindi speaker but a proficient (or semi-proficient) Hindi speaker, the main results of our analysis should hold.…”

Section: Re-annotation Resultssupporting

confidence: 60%

“…The native language L 1 is Hindi and the foreign language L 2 is English. To bootstrap the data collection process, we used the language tagged tweets presented in (Rudra et al, 2016). In addition to this, we also crawled tweets (between Nov 2015 and Jan 2016) related to 28 hashtags representing different Indian contexts covering important topics such as sports, religion, movies, politics etc.…”

Section: Datasets and Preprocessingmentioning

confidence: 99%

See 1 more Smart Citation

All that is English may be Hindi: Enhancing language identification through automatic ranking of the likeliness of word borrowing in social media

Patro¹,

Samanta²,

Singh³

et al. 2017

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing

Self Cite

View full text Add to dashboard Cite

In this paper, we present a set of computational methods to identify the likeliness of a word being borrowed, based on the signals from social media. In terms of Spearman's correlation values, our methods perform more than two times better (∼ 0.62) in predicting the borrowing likeliness compared to the best performing baseline (∼ 0.26) reported in literature. Based on this likeliness estimate we asked annotators to re-annotate the language tags of foreign words in predominantly native contexts. In 88% of cases the annotators felt that the foreign language tag should be replaced by native language tag, thus indicating a huge scope for improvement of automatic language identification systems.

show abstract

Section: Re-annotation Resultssupporting

confidence: 60%

Section: Datasets and Preprocessingmentioning

confidence: 99%

All that is English may be Hindi: Enhancing language identification through automatic ranking of the likeliness of word borrowing in social media

Patro¹,

Samanta²,

Singh³

et al. 2017

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing

Self Cite

View full text Add to dashboard Cite

show abstract

“…Generating plausible code-switched text is an even more delicate task than generating monolingual text. Linguistic studies show that bilingual speakers switch languages by following various complex constraints [18,17] which may even include the intensity of sentiment expressed in various segments of text [23]. [20] synthesized code-mixed sentences by leveraging linguistic constraints arising from Equivalence Constraint Theory.…”

Section: Introductionmentioning

confidence: 99%

A Deep Generative Model for Code Switched Text

Samanta

Reddy

Jagirdar

et al. 2019

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

Code-switching, the interleaving of two or more languages within a sentence or discourse is pervasive in multilingual societies. Accurate language models for code-switched text are critical for NLP tasks. Stateof-the-art data-intensive neural language models are difficult to train well from scarce language-labeled code-switched text. A potential solution is to use deep generative models to synthesize large volumes of realistic code-switched text. Although generative adversarial networks and variational autoencoders can synthesize plausible monolingual text from continuous latent space, they cannot adequately address codeswitched text, owing to their informal style and complex interplay between the constituent languages. We introduce VACS, a novel variational autoencoder architecture specifically tailored to code-switching phenomena. VACS encodes to and decodes from a two-level hierarchical representation, which models syntactic contextual signals in the lower level, and language switching signals in the upper layer. Sampling representations from the prior and decoding them produced well-formed, diverse code-switched sentences. Extensive experiments show that using synthetic code-switched text with natural monolingual data results in significant (33.06%) drop in perplexity.

show abstract

“…Sentiment analysis on social media is critical for commerce and governance. Multilingual social media users often use code-switching, particularly to express emotion [28]. However, a basic requirement to train any sentiment analysis (SA) system is the availability of large sentiment-labeled corpora.…”

Section: Introductionmentioning

confidence: 99%

Improved Sentiment Detection via Label Transfer from Monolingual to Synthetic Code-Switched Text

Samanta¹,

Ganguly²,

Chakrabarti³

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

Multilingual writers and speakers often alternate between two languages in a single discourse, a practice called "code-switching". Existing sentiment detection methods are usually trained on sentiment-labeled monolingual text. Manually labeled code-switched text, especially involving minority languages, is extremely rare. Consequently, the best monolingual methods perform relatively poorly on code-switched text. We present an effective technique for synthesizing labeled code-switched text from labeled monolingual text, which is more readily available. The idea is to replace carefully selected subtrees of constituency parses of sentences in the resource-rich language with suitable token spans selected from automatic translations to the resource-poor language. By augmenting scarce humanlabeled code-switched text with plentiful synthetic code-switched text, we achieve significant improvements in sentiment labeling accuracy (1.5%, 5.11%, 7.20%) for three different language pairs (English-Hindi, English-Spanish and English-Bengali). We also get significant gains for hate speech detection: 4% improvement using only synthetic text and 6% if augmented with real text.

show abstract

Understanding Language Preference for Expression of Opinion and Sentiment: What do Hindi-English Speakers do on Twitter?

Cited by 57 publications

References 27 publications

All that is English may be Hindi: Enhancing language identification through automatic ranking of the likeliness of word borrowing in social media

All that is English may be Hindi: Enhancing language identification through automatic ranking of the likeliness of word borrowing in social media

A Deep Generative Model for Code Switched Text

Improved Sentiment Detection via Label Transfer from Monolingual to Synthetic Code-Switched Text

Contact Info

Product

Resources

About