A Complete Process of Text Classification System Using State-of-the-Art NLP Models

Abstract

With the rapid advancement of information technology, online information has been exponentially growing day by day, especially in the form of text documents such as news events, company reports, reviews on products, stocks-related reports, medical reports, tweets, and so on. Due to this, online monitoring and text mining has become a prominent task. During the past decade, significant efforts have been made on mining text documents using machine and deep learning models such as supervised, semisupervised, and unsupervised. Our area of the discussion covers state-of-the-art learning models for text mining or solving various challenging NLP (natural language processing) problems using the classification of texts. This paper summarizes several machine learning and deep learning algorithms used in text classification with their advantages and shortcomings. This paper would also help the readers understand various subtasks, along with old and recent literature, required during the process of text classification. We believe that readers would be able to find scope for further improvements in the area of text classification or to propose new techniques of text classification applicable in any domain of their interest.

Biomedical Text Classification Using Augmented Word Representation Based on Distributional and Relational Contexts

“…Researchers have demonstrated the adaptability of Word2Vec and BERT in the feld of biomedical domain to develop models such as BioWordVec [33] and BioBERT [34], as well as other domain-specifc models such as SciBERT [35] trained on various scientifc and biomedical corpuses, ClinicalBERT [36] trained on clinical notes for various NLP tasks, and MatSciBERT [37] trained on material science publications. Deep learning models that take such trained word representations as input have been employed by researchers to classify unstructured texts documents [38], medical notes [39], health-related social media texts [40], and biomedical text mining tasks [41]. Besides these, handwritten script recognition [42], detection of diseases [43][44][45], and healthcare solutions [46] involve the potential application of deep learning models.…”

Mengoptimalkan Akurasi pada Klasifikasi Emosi Majemuk Berdasarkan Semantik Kalimat Menggunakan XLM-RoBERTa

“…The model only required five times training using the loss function binary cross-entropy (BCE) [27] using formula (4).…”

“…Six labels in a text that correspond to the basic emotions are used in the classification process [3]. Various techniques, including support vector machine (SVM), naïve Bayes, random forest, convolutional neural networks, have been used in numerous prior research [4]- [7]. The cross-lingual language modelrobustly optimized bidirectional encoder representations from transformers approach (XLM-RoBERTa) model could improve the classification performance of a hate speech text in Indonesian to 89.52%, compared with the previous research using long short term memory (LSTM), which only reached 77.36% optimization [1].…”

An Assessment of Lexical, Network, and Content-Based Features for Detecting Malicious URLs Using Machine Learning and Deep Learning Models

“…The average achieved accuracy was 83.21%, the average F-positive rate was 10.03%, and the average F-measure was 86%. Similarly, Dogra et al [ 48 ] also considered the pattern of the URLs posting in twitter social media platforms by analyzing the behavior of the URLs posting users and URLs clicking users. Using twitter APIs combined with Bitly APIs they collected around 7 million tweets that contain shortened URLs created by Bitly and tried different sets of features including Average clicks, Posting count, Median followers, Median friends, Score function Score Category.…”