Predicting Code-switching in Multilingual Communication for Immigrant Communities

Abstract

Immigrant communities host multilingual speakers who switch across languages and cultures in their daily communication practices. Although there are in-depth linguistic descriptions of code-switching across different multilingual communication settings, there is a need for automatic prediction of code-switching in large datasets. We use emoticons and multi-word expressions as novel features to predict code-switching in a large online discussion forum for the Turkish-Dutch immigrant community in the Netherlands. Our results indicate that multi-word expressions are powerful features to predict code-switching.

Word-Level Language Identification and Predicting Codeswitching Points in Swahili-English Language Data

“…Relative to previous work on codeswitch prediction, our F1 score is similar but higher than in Solorio and Liu (2008) in the unbalanced condition. In the balanced condition, our F1 scores are similar to those reported in Papalexakis et al (2014) on the interview data.…”

“…The amount of codeswitching appears to be fairly similar, despite the difference in language distribution. Previous approaches to this problem have used naïve Bayes classifiers trained using contextual and POS features (Solorio and Liu, 2008;Papalexakis et al, 2014). We explored a similar set of features, but additionally tried to represent a few other intuitions.…”

“…For features 6-10, we used binning to make the values more appropriate for the naïve Bayes algorithm. Features 1-3 and 11 were previously used in either Solorio and Liu (2008), Papalexakis et al (2014), or both and found to have predictive value. Features 4 and 5 are meant as alternative versions of 2 and 3.…”

“…We could only find two articles that deal precisely with this task, Solorio and Liu (2008) and Papalexakis, Nguyen and Dogruöz (2014). The two groups take fairly different approaches to feature design and performance evaluation, while both groups use naïve Bayes classifiers.…”

“…on the full set. In the balanced condition, random samples of approximately 10,000 switch points and 10,000 non-switch points were taken from the data sets in a manner similar to Papalexakis et al (2014). This allows a more direct comparison to both previous papers.…”