A Comparative Study of Some Automatic Arabic Text Diacritization Systems

Abstract

Arabic diacritization is the task of restoring diacritics or vowels for Arabic texts considering that they are mostly written without them. This task, when automated, shows better results for some natural language processing tasks; hence, it is necessary for the field of Arabic language processing. In this paper, we are going to present a comparative study of some automatic diacritization systems. One uses a variant of the hidden Markov model. The other one is a pipeline, which includes a Long Short-Term Memory deep learning model, a rule-based correction component, and a statistical-based component. Additionally, we are proposing some modifications to those systems. We have trained and tested those systems in the same benchmark dataset based on the same evaluation metrics proposed in previous work. The best system results are 9.42% and 22.82% for the diacritic error rate DER and the word error rate WER, respectively.

Search and Comparison of Isim Ma'rifat with Remove Diacritic in the Qur'an and Hadith of Abu Daud

“…Diacritical punctuation or vowels are written above or below the letters. It should be noted that the speed of typing in Arabic script can be halved if we include diacritics in the written text [14]. The diacritic removal algorithm works to remove diacritical punctuation marks or vowels.…”