Impact of Stemming and Word Embedding on Deep Learning-Based Arabic Text Categorization

Abstract

Document classification is a classical problem in information retrieval, and plays an important role in a variety of applications. Automatic document classification can be defined as content-based assignment of one or more predefined categories to documents. Many algorithms have been proposed and implemented to solve this problem in general, however, classifying Arabic documents is lagging behind similar works in other languages. In this paper, we present seven deep learning-based algorithms to classify the Arabic documents. These are: Convolutional Neural Network (CNN), CNN-LSTM (LSTM = Long Short-Term Memory), CNN-GRU (GRU = Gated Recurrent Units), BiLSTM (Bidirectional LSTM), BiGRU, Att-LSTM (Attention-based LSTM), and Att-GRU. And for word representation, we applied the word embedding technique (Word2Vec). We tested our approach on two large datasets-with six and eight categories-using ten-fold cross-validation. Our objective was to study how the classification is affected by the stemming strategies and word embedding. First, we looked into the effects of different stemming algorithms on the document classification with different deep learning models. We experimented with eleven different stemming algorithms, broadly falling into: root-based and stem-based, and no stemming. We performed ANOVA test on the classification results using the different stemmers, which helps assure if the results are significant. The results of our study indicate that stem-based algorithms perform slightly better compared to root-based algorithms. Among the deep learning models, the Attention mechanism and the Bidirectional learning gave outstanding performance with Arabic text categorization. Our best performance is F -score = 97.96%, achieved using the Att-GRU model with stem-based algorithm. Next, we looked into different controlling parameters for word embedding. For Word2Vec, both skip-gram and bag-of-words (CBOW) perform well with either stemming strategies. However, when using a stem-based algorithm, skipgram achieves good results with a vector of smaller dimension, while CBOW requires a larger dimension vector to achieve a similar performance.

Effect of stemming on text similarity for Arabic language at sentence level

“…Preprocessing is a key task in semantic text similarity process. Stemming is an important technique adopted for preprocessing texts due to the fact that it reduces feature space and improves performance of the similarity process ( Alhaj et al, 2019 ; Almuzaini & Azmi, 2020 ).…”

“…Stemming effect has been studied and applied to different domains of NLP and computation linguistics. This includes document categorization ( Alhaj et al, 2019 ; Almuzaini & Azmi, 2020 ), information retrieval ( Zeroual & Lakhouaja, 2017 ; Alnaied, Elbendak & Bulbul, 2020 ), automatic essay scoring ( Al-Shalabi, 2016 ), and sentiment analysis ( Al-Saqqa, Awajan & Ghoul, 2019 ). In all these studies it has been reported that stemming and lemmatization improves the performance of the resulted models.…”

Analisa Modifikasi Algoritma Stemming Untuk Kasus Overstemming

“…Pengujian dilakukan dengan mengolah atribut judul tersebut melalui proses stemming biasa dan proses modifikasi stemming. Kedua proses tersebut akan menghasilkan perbandingan nilai recall pada judul [10], [11].…”

Deep Sentiment Analysis: A Case Study on Stemmed Turkish Twitter Data

“…The stemming rather reduces the information gained from the data in many languages. In fact, the stemming improves accuracy (ACC [28]) achieved by various methods in different languages including not only English [29], [30] but also Arabic [26], [27], [31], [32], Indonesian [23], [33], [34], Japanese [25], [35] French [36]- [38], Portuguese [37], [39], German [37], [40], [41], Hungarian [37], [42], [43], Spanish [44]- [47], and Turkish [48]- [50].…”