Addressing Unseen Word Problem in Text Classification

Yenigalla, Promod; Kar, S.; Singh, Chirag; Nagar, Ajay; Mathur, Gaurav

doi:10.1007/978-3-319-91947-8_36

Cited by 10 publications

(3 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It increased a small number of accuracy but took a very long time due to a huge training corpus. The same problem of the unseen word was addressed in [51]. The researchers proposed to use CNN and combine both character and word-based models to efficiently perform text classification.…”

Section: Related Workmentioning

confidence: 99%

Impact of convolutional neural network and FastText embedding on text classification

Umer

Imtiaz

Nappi

et al. 2022

Multimed Tools Appl

View full text Add to dashboard Cite

Efficient word representation techniques (word embeddings) with modern machine learning models have shown reasonable improvement on automatic text classification tasks. However, the effectiveness of such techniques has not been evaluated yet in terms of insufficient word vector representation for training. Convolutional Neural Network has achieved significant results in pattern recognition, image analysis, and text classification. This study investigates the application of the CNN model on text classification problems by experimentation and analysis. We trained our classification model with a prominent word embedding generation model, Fast Text on publically available datasets, six benchmark datasets including Ag News, Amazon Full and Polarity, Yahoo Question Answer, Yelp Full, and Polarity. Furthermore, the proposed model has been tested on the Twitter US airlines non-benchmark dataset as well. The analysis indicates that using Fast Text as word embedding is a very promising approach.

show abstract

Section: Related Workmentioning

confidence: 99%

Impact of convolutional neural network and FastText embedding on text classification

Umer

Imtiaz

Nappi

et al. 2022

Multimed Tools Appl

View full text Add to dashboard Cite

show abstract

“…Using trained word-embedding models provides an opportunity for the classifier to correctly classify words that were not seen in the training data set [ 118 ], which solves the problem in traditional text classification that occurs when the classifier fails upon encountering an unseen word [ 119 ]. For pre-processing text in the second experiment, we employed the same steps provided by the authors of pre-trained word embeddings models.…”

Section: Second Experimentsmentioning

confidence: 99%

Investigating the impact of pre-processing techniques and pre-trained word embeddings in detecting Arabic health information on social media

2021

View full text Add to dashboard Cite

This paper presents a comprehensive evaluation of data pre-processing and word embedding techniques in the context of Arabic document classification in the domain of health-related communication on social media. We evaluate 26 text pre-processings applied to Arabic tweets within the process of training a classifier to identify health-related tweets. For this task we use the (traditional) machine learning classifiers KNN, SVM, Multinomial NB and Logistic Regression. Furthermore, we report experimental results with the deep learning architectures BLSTM and CNN for the same text classification problem. Since word embeddings are more typically used as the input layer in deep networks, in the deep learning experiments we evaluate several state-of-the-art pre-trained word embeddings with the same text pre-processing applied. To achieve these goals, we use two data sets: one for both training and testing, and another for testing the generality of our models only. Our results point to the conclusion that only four out of the 26 pre-processings improve the classification accuracy significantly. For the first data set of Arabic tweets, we found that Mazajak CBOW pre-trained word embeddings as the input to a BLSTM deep network led to the most accurate classifier with F1 score of 89.7%. For the second data set, Mazajak Skip-Gram pre-trained word embeddings as the input to BLSTM led to the most accurate model with F1 score of 75.2% and accuracy of 90.7% compared to F1 score of 90.8% achieved by Mazajak CBOW for the same architecture but with lower accuracy of 70.89%. Our results also show that the performance of the best of the traditional classifier we trained is comparable to the deep learning methods on the first dataset, but significantly worse on the second dataset.

show abstract

“…Correspondingly, for the text, local features are sliding windows consisting of several words, similar to N-grams. The advantage of Convolutional Neural Networks is that they can automatically combine and filter N-gram features to obtain local semantic information at different levels of abstraction [ 41 , 42 , 43 , 44 , 45 , 46 ]. Then, the maximum pooling operation is applied to the feature mapping to obtain the maximum value as input to the Transformer layer.…”

Section: Related Work and Backgroundmentioning

confidence: 99%

Improving Ponzi Scheme Contract Detection Using Multi-Channel TextCNN and Transformer

Dai¹,

et al. 2021

Sensors

View full text Add to dashboard Cite

With the development of blockchain technologies, many Ponzi schemes disguise themselves under the veil of smart contracts. The Ponzi scheme contracts cause serious financial losses, which has a bad effect on the blockchain. Existing Ponzi scheme contract detection studies have mainly focused on extracting hand-crafted features and training a machine learning classifier to detect Ponzi scheme contracts. However, the hand-crafted features cannot capture the structural and semantic feature of the source code. Therefore, in this study, we propose a Ponzi scheme contract detection method called MTCformer (Multi-channel Text Convolutional Neural Networks and Transofrmer). In order to reserve the structural information of the source code, the MTCformer first converts the Abstract Syntax Tree (AST) of the smart contract code to the specially formatted code token sequence via the Structure-Based Traversal (SBT) method. Then, the MTCformer uses multi-channel TextCNN (Text Convolutional Neural Networks) to learn local structural and semantic features from the code token sequence. Next, the MTCformer employs the Transformer to capture the long-range dependencies of code tokens. Finally, a fully connected neural network with a cost-sensitive loss function in the MTCformer is used for classification. The experimental results show that the MTCformer is superior to the state-of-the-art methods and its variants in Ponzi scheme contract detection.

show abstract

Addressing Unseen Word Problem in Text Classification

Cited by 10 publications

References 11 publications

Impact of convolutional neural network and FastText embedding on text classification

Impact of convolutional neural network and FastText embedding on text classification

Investigating the impact of pre-processing techniques and pre-trained word embeddings in detecting Arabic health information on social media

Improving Ponzi Scheme Contract Detection Using Multi-Channel TextCNN and Transformer

Contact Info

Product

Resources

About