Abstract:A growing amount of research use pre‐trained language models to address few/zero‐shot text classification problems. Most of these studies neglect the semantic information hidden implicitly beneath the natural language names of class labels and develop a meta learner from the input texts solely. In this work, we demonstrate how label information can be utilized to extract enhanced feature representation of the input text from a Transformer‐based pre‐trained language model such as AraBERT. In addition, how this … Show more
Text classification is the process of labelling a given set of text documents with predefined classes or categories. Existing Arabic text classifiers are either applying classic Machine Learning algorithms such as k‐NN and SVM or using modern deep learning techniques. The former are assessed using small text collections and their accuracy is still subject to improvement while the latter are efficient in classifying big data collections and show limited effectiveness in classifying small corpora with a large number of categories. This paper proposes a new approach to Arabic text classification to treat small and large data collections while improving the classification rates of existing classifiers. We first demonstrate the ability of analogical proportions (AP) (statements of the form ‘x is to as is to ’), which have recently been shown to be effective in classifying ‘structured’ data, to classify ‘unstructured’ text documents requiring preprocessing. We design an analogical model to express the relationship between text documents and their real categories. Next, based on this principle, we develop two new analogical Arabic text classifiers. These rely on the idea that the category of a new document can be predicted from the categories of three others, in the training set, in case the four documents build together a ‘valid’ analogical proportion on all or on a large number of components extracted from each of them. The two proposed classifiers (denoted AATC1 and AATC2) differ mainly in terms of the keywords extracted for classification. To evaluate the proposed classifiers, we perform an extensive experimental study using five benchmark Arabic text collections with small or large sizes, namely ANT (Arabic News Texts) v2.1 and v1.1, BBC‐Arabic, CNN‐Arabic and AlKhaleej‐2004. We also compare analogical classifiers with both classical ML‐based and Deep Learning‐based classifiers. Results show that AATC2 has the best average accuracy (78.78%) over all other classifiers and the best average precision (0.77) ranked first followed by AATC1 (0.73), NB (0.73) and SVM (0.72) for the ANT corpus v2.1. Besides, AATC1 shows the best average precisions (0.88) and (0.92), respectively for the BBC‐Arabic corpus and AlKhaleej‐2004, and the best average accuracy (85.64%) for CNN‐Arabic over all other classifiers. Results demonstrate the utility of analogical proportions for text classification. In particular, the proposed analogical classifiers are shown to significantly outperform a number of existing Arabic classifiers, and in many cases, compare favourably to the robust SVM classifier.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.