Arabic Transliteration of Romanized Tunisian Dialect Text: A Preliminary Investigation

Abstract

Abstract. In this paper, we describe the process of converting Tunisian Dialect text that is written in Latin script (also called Arabizi) into Arabic script following the CODA orthography convention for Dialectal Arabic. Our input consists of messages and comments taken from SMS, social networks and broadcast videos. The language used in social media and SMS messaging is characterized by the use of informal and non-standard vocabulary such as repeated letters for emphasis, typos, non-standard abbreviations, and nonlinguistic content, such as emoticons. There is a high degree of variation is spelling in Arabic dialects due to the lack of orthographic widely supported standards in both Arabic and Latin scripts. In the context of natural language processing, transliterating from Arabizi to Arabic script is a necessary step since most recently available tools for processing Arabic Dialects expect Arabic script input.

Contributions to the automatic processing of the user-generated Tunisian dialect on the social web

“…The transliteration task was tackled by Masmoudi et al (2015) who resorted to the CODA standard for the TD (Zribi et al, 2014), and by Younes et al (2016) who used a machine learning approach, based on hidden Markov models (HMM). Both works focused on the transliteration of the Latin form of the TD into Arabic script.…”

“…This form of writing, also referred to as 'Arabizi' and 'Romanised', is marked by several phenomena such as the use of digits to replace the Arabic letters with no equivalents in the Latin alphabet, abbreviations, acronyms, the use of vowels, etc., presenting thus, additional challenges and difficulties to overcome. All though this form of TD writing is more present on the social web, only few works focused on constructing LRs for the Romanised TD, namely Masmoudi et al (2015) and Younes et al (2015Younes et al ( , 2016.…”

“…This is why many researches in the natural language processing (NLP) field have been carried out in recent years, in order to deal with the ADs, especially when they are written on the social web. Some of them focused on constructing corresponding language resources (LRs) (Diab et al, 2000;Zaidan and Callison-Burch, 2011;Meftouh et al, 2012;Boujelbane et al, 2013a;Bouchlaghem et al, 2014;Younes and Souissi, 2014), others tackled their automatic processing (Al-Sabbagh and Girjuh, 2012;Chalabi and Gergers, 2012;Darwish, 2014;Masmoudi et al, 2015;Younes et al, 2016;Aridhi et al, 2017).…”

Natural Language Processing for Dialectical Arabic: A Survey

“…Recently, a conventional orthography for Tunisian Arabic was proposed in (Zribi et al, 2014). Also, Several papers on the transliteration from Arabizi into Arabic orthography appeared (Bies et al, 2014), (Darwish, 2013), (Masmoudi et al, 2015). Arabizi is Arabic text written in Latin characters.…”