Complexity Metric for Code-Mixed Social Media Text

Abstract

Abstract. An evaluation metric is an absolute necessity for measuring the performance of any system and complexity of any data. In this paper, we have discussed how to determine the level of complexity of code-mixed social media texts that are growing rapidly due to multilingual interference. In general, texts written in multiple languages are often hard to comprehend and analyze. At the same time, in order to meet the demands of analysis, it is also necessary to determine the complexity of a particular document or a text segment. Thus, in the present paper, we have discussed the existing metrics for determining the code-mixing complexity of a corpus, their advantages and shortcomings as well as proposed several improvements on the existing metrics. The new index better reflects the variety and complexity of a multilingual document. Also, the index can be applied to a sentence and seamlessly extended to a paragraph or an entire document. We have employed two existing code-mixed corpora to suit the requirements of our study.

CESAR: A New Metric to Measure the Level of Code-Switching in Corpora - Application to Maghrebian Dialects

“…In this article we measure the complexity of code-switching of each of the three studied languages by using the metric proposed in [14]. This measure is not adapted to what we would like to quantify.…”

“…In Section 4, we discuss the proposed CESAR metric that measures the complexity of codeswitching in multilingual texts. We evaluate this measure and compare it against the one proposed by [14] in Section 5 we conclude.…”

“…collected from social networks. In [14], the authors proposed another measure to determine the level of code-switching. This metric combines three parameters: a language factor that indicates the number of languages present simultaneously in a text, a mixed factor which is related to the number of words not written in the dominant language and a switched factor that corresponds to the number of language break points.…”

“…In other words, we aim to determine how much a document is clean relatively to a reference language. The scores proposed in [9] and [14] are not suitable for our purpose since they calculate the complexity factor (CF) of a corpus independently from any language. They are interested in the phenomenon of code-switching without paying attention to the tongue mother of the Internet user.…”

Should Code-switching Models Be Asymmetric?

“…While asymmetrical models have been widely assumed in NLP research [1,8,9,10,11,12,13,14,15,16,17,18,19,20,21], Bhat et al (2016) finds the MLF model to be incomplete and unsound, without a complementary symmetrical model [5], for building a working model of C-S [22]. They correctly point out that the MLF model is underspecified with respect to the categories that can or cannot be switched, a criticism that has also been leveled by linguists.…”