Moroccan Data-Driven Spelling Normalization Using Character Neural Embedding

Abstract

With the increase of Web use in Morocco today, Internet has become an important source of information. Specifically, across social media, the Moroccan people use several languages in their communication leaving behind unstructured user-generated text (UGT) that presents several opportunities for Natural Language Processing. Among the languages found in this data, Moroccan Arabic (MA) stands with an important content and several features. In this paper, we investigate online written text generated by Moroccan users in social media with an emphasis on Moroccan Arabic. For this purpose, we follow several steps, using some tools such as a language identification system, in order to conduct a deep study of this data. The most interesting findings that have emerged are the use of code-switching, multi-script and low amount of words in the Moroccan UGT. Moreover, we used the investigated data in order to build a new Moroccan language resource. The latter consists in building a Moroccan words orthographic variants lexicon following an unsupervised approach and using character neural embedding. This lexicon can be useful for several NLP tasks such as spelling normalization.

Building a Corpus for the Underexplored Moroccan Dialect (CFMD) Through Audio Segmentations

“…The Moroccan dialect is influenced by foreign words from the French or Spanish language [5]. To a greater or a lesser degree 11% from French, and less than 1% from both Spanish and Tamazight languages [18,12,16]. Nowadays, Morocco has reached approximately 37 million people divided into 12 regions of the kingdom.…”

Non-Conventional Spelling in Informal, Colloquial Arabic Writing on Facebook

“…This system is comprised of an orthographic variants lexicon of Moroccan words that utilize an unsupervised approach and character neural embedding. This lexicon can be useful for several natural language processing tasks such as spelling normalization (Tachicart & Bouzoubaa, 2021) Finally, the impact of different types of social media on Arabic-speaking students' reading, and writing skill development, and their impact on the students' spelling and language use in written schoolwork is still open for further investigation in the future.…”