Multi-class Text Classification Model Based on Weighted Word Vector and BiLSTM-Attention Optimization

Wu, Hao; He, Zhuangzhuang; Zhang, Weitao; Hu, Yunsheng; Wu, Yunzhi; Yue, Yi

doi:10.1007/978-3-030-84522-3_32

Cited by 9 publications

(4 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The obtained results showed that Bi-LSTM outperforms the traditional techniques, provides a better capture of context information, and has higher precision, recall and F1 than the other methods. Wu et al (2021) demonstrated in their research that Bi-LSTM units can effectively solve the gradient problems and can well capture the contextual semantic information.…”

Section: Related Workmentioning

confidence: 99%

Social media-based COVID-19 sentiment classification model using Bi-LSTM

Arbane¹,

Benlamri²,

Brik³

et al. 2023

Expert Systems with Applications

View full text Add to dashboard Cite

Section: Related Workmentioning

confidence: 99%

Social media-based COVID-19 sentiment classification model using Bi-LSTM

Arbane¹,

Benlamri²,

Brik³

et al. 2023

Expert Systems with Applications

View full text Add to dashboard Cite

“…Considering the fact that adding more hidden layers to BERT will not improve its performance, the authors have added a Bi-LSTM layer to each of the transformer entity, called TRANS-BLSTM, and have observed that their model provides an F1-score of 94.01% on SQUAD 1.1 development dataset. Hao Wu et al [11] have proposed a weighted multi-class text classification model where the text is converted to its numerical terms using Word2Vec technique; weights are applied to those vectors using TF-IDF algorithm, and the word vectors are multiplied with these weights to provide the final representation of text. Context is captured using a BiLSTM layer, followed by an Attention layer and a softmax layer to classify.…”

Section: Literature Surveymentioning

confidence: 99%

BERT-Based Hybrid RNN Model for Multi-class Text Classification to Study the Effect of Pre-trained Word Embeddings

Shreyashree¹,

Sunagar²,

Rajarajeswari³

et al. 2022

IJACSA

View full text Add to dashboard Cite

Due to the Covid-19 pandemic which started in the year 2020, many nations had imposed lockdown to curb the spread of this virus. People have been sharing their experiences and perspectives on social media on the lockdown situation. This has given rise to increased number of tweets or posts on social media. Multi-class text classification, a method of classifying a text into one of the pre-defined categories, is one of the effective ways to analyze such data that is implemented in this paper. A Covid-19 dataset is used in this work consisting of fifteen predefined categories. This paper presents a multi-layered hybrid model, LSTM followed by GRU, to integrate the benefits of both the techniques. The advantages of word embeddings techniques like GloVe and BERT have been implemented and found that, for three epochs, the transfer learning based pre-trained BERThybrid model performs one percent better than GloVe-hybrid model but the state-of-the-art, fine-tuned BERT-base model outperforms the BERT-hybrid model by three percent, in terms of validation loss. It is expected that, over a larger number of epochs, the hybrid model might outperform the fine-tuned model.

show abstract

“…There are various approaches to Natural Language Processing (NLP), including [9]that using the LSTM method to classify the sentiment of online coronavirus discussion topics, Classification of COVID-19 related tweets in Nepal using a set of CNN's [10]. This research will use the latest development algorithm of neural networks, namely Bidirectional Long Short-Term Memory (BiLSTM) in making the final project title classification model because from research [11]has shown that the Bi-LSTM method is one of the most successful methods in the context of text classification by comparing the use of Bi-LSTM in sentiment analysis with neural network (RNN), convolutional neural network (CNN), traditional LSTM, and Naïve Bayes (NB) methods. The results show that Bi-LSTM outperforms the other methods, producing better contextual information and higher precision, recall and F1 than other methods.…”

Section: Introductionmentioning

confidence: 99%

Classification of Final Project Titles Using Bidirectional Long Short Term Memory at the Faculty of Engineering Nurul Jadid University

Warda,

Fajri,

Tholib

2023

SISFOKOM

View full text Add to dashboard Cite

Every year, the Faculty of Engineering at Nurul Jadid University forms a committee to manage the process of students' final projects from the title selection stage to the final examination process until graduation. The process of selecting the final project title is still done manually, namely by checking the titles one by one, which takes a long time and allows errors because there is a lot of data to check, so human errors can also occur. Therefore, this research proposes to use the Bidirectional Long Short Term Memory (BiLSTM) method to classify the final project title based on its grade category. Several experiments were conducted to generate the most appropriate labels. The first experiment produced 4 labels and the second experiment produced 2 labels. From the results of several experiments, it was concluded that the second experiment had the best accuracy results with the 'good enough' and 'good' classes. The oversampling technique was then applied to overcome overlapping data, and the turning process was then performed on several parameters that could re-optimize the previous accuracy result of 75.24% to 91.15%. With a configuration of 10 random state parameters, using 64 batch sizes and 50 epochs. In addition, model adjustments were made to the hidden layer by adding a dropout layer and relu activation.

show abstract

Multi-class Text Classification Model Based on Weighted Word Vector and BiLSTM-Attention Optimization

Cited by 9 publications

References 6 publications

Social media-based COVID-19 sentiment classification model using Bi-LSTM

Social media-based COVID-19 sentiment classification model using Bi-LSTM

BERT-Based Hybrid RNN Model for Multi-class Text Classification to Study the Effect of Pre-trained Word Embeddings

Classification of Final Project Titles Using Bidirectional Long Short Term Memory at the Faculty of Engineering Nurul Jadid University

Contact Info

Product

Resources

About