MultiLexNorm: A Shared Task on Multilingual Lexical Normalization

Abstract

Lexical normalization is the task of transforming an utterance into its standardized form. This task is beneficial for downstream analysis, as it provides a way to harmonize (often spontaneous) linguistic variation. Such variation is typical for social media on which information is shared in a multitude of ways, including diverse languages and code-switching. Since the seminal work of Han and Baldwin (2011) a decade ago, lexical normalization has attracted attention in English and multiple other languages. However, there exists a lack of a common benchmark for comparison of systems across languages with a homogeneous data and evaluation setup. The MUL-TILEXNORM shared task sets out to fill this gap. We provide the largest publicly available multilingual lexical normalization benchmark including 12 language variants. We propose a homogenized evaluation setup with both intrinsic and extrinsic evaluation. As extrinsic evaluation, we use dependency parsing and part-ofspeech tagging with adapted evaluation metrics (a-LAS, a-UAS, and a-POS) to account for alignment discrepancies. The shared task hosted at W-NUT 2021 attracted 9 participants and 18 submissions. The results show that neural normalization systems outperform the previous state-of-the-art system by a large margin. Downstream parsing and part-of-speech tagging performance is positively affected but to varying degrees, with improvements of up to 1.72 a-LAS, 0.85 a-UAS, and 1.54 a-POS for the winning system. 1

Addressing religious hate online: from taxonomy creation to automated detection

“…Interesting future avenues for research include also studying the impact of lexical normalization on downstream abusive language detection and religious hate speech detection performance, using monolingual ( van der Goot et al, 2020 ; Baldwin et al, 2015 ) or multilingual datasets ( van der Goot et al, 2021a ), as well as exploiting multiple annotations on the Italian portion of the dataset to study intersectionality.…”

Term Standardisation With LDA Model To Detect Service Disruption Events Using English And Manglish Tweets

“…However, in this study, a simpler technique was used, which involved term standardisation. Term standardisation assures homogeneity and harmonisation throughout the texts and decreases the needed efforts for further text processing [34,35,36]. In addition, through the standardisation process, synonyms, slang, abbreviations, and other related aspects can be standardised, which potentially enhances the ability of LDA in identifying topic terms as LDA considers the distribution and frequency of words in the documents.…”

One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia

“…Some multilingual datasets for question answering (TyDiQA; Clark et al, 2020), common sense reasoning (XCOPA;Ponti et al, 2020), abstractive summarization (Hasan et al, 2021), passage ranking (mMARCO;Bonifacio et al, 2021), crosslingual visual question answering (xGQA;Pfeiffer et al, 2021), language and vision reasoning (MaRVL; Liu et al, 2021), paraphrasing (Para-Cotta;, dialogue systems (XPersona & BiToD; Lin et al, 2021a,b), lexical normalization (MultiLexNorm;van der Goot et al, 2021), and machine translation (FLORES-101; Guzmán et al, 2019) include Indonesian but most others do not, and very few include Indonesian local lan-guages. An exception is the weakly supervised named entity recognition dataset, WikiAnn (Pan et al, 2017), which covers several Indonesian local languages, namely Acehnese, Javanese, Minangkabau, and Sundanese.…”