AB-LaBSE: Uyghur Sentiment Analysis via the Pre-Training Model with BiLSTM

Abstract

In recent years, more and more attention has been paid to text sentiment analysis, which has gradually become a research hotspot in information extraction, data mining, Natural Language Processing (NLP), and other fields. With the gradual popularization of the Internet, sentiment analysis of Uyghur texts has great research and application value in online public opinion. For low-resource languages, most state-of-the-art systems require tens of thousands of annotated sentences to get high performance. However, there is minimal annotated data available about Uyghur sentiment analysis tasks. There are also specificities in each task—differences in words and word order across languages make it a challenging problem. In this paper, we present an effective solution to providing a meaningful and easy-to-use feature extractor for sentiment analysis tasks: using the pre-trained language model with BiLSTM layer. Firstly, data augmentation is carried out by AEDA (An Easier Data Augmentation), and the augmented dataset is constructed to improve the performance of text classification tasks. Then, a pretraining model LaBSE is used to encode the input data. Then, BiLSTM is used to learn more context information. Finally, the validity of the model is verified via two categories datasets for sentiment analysis and five categories datasets for emotion analysis. We evaluated our approach on two datasets, which showed wonderful performance compared to some strong baselines. We close with an overview of the resources for sentiment analysis tasks and some of the open research questions. Therefore, we propose a combined deep learning and cross-language pretraining model for two low resource expectations.

Document-Level Sentiment Analysis Using Attention-Based Bi-Directional Long Short-Term Memory Network and Two-Dimensional Convolutional Neural Network

“…The performance of the model is evaluated using accuracy (higher is better), F1-Measure (higher is better), and MSE (lower is better). The results of certain based methods were referenced from previous studies [13,18,35,41], and we obtained other methods' results through experiments. Based on the results in Table 2, multiple findings are obtained.…”

“…It is commonly appreciated that accuracy is a standard metric used to evaluate the overall sentiment analysis performance [38][39][40][41][42]. According to the research by Pei et al [41], Rao et al [16], and Behera et al [43], this paper adds two evaluation parameters (F1-Measureand MSE) to evaluate the performance of sentiment analysis. Finally, we employ Accuracy, F1-Measure, and MSE to assess our model in this paper.…”

Hyperparamaters Fine Tuning for Bidirectional Long Short Term Memory on Food Delivery

“…For datasets from hotel reviews, it gets an accuracy of 85.66%, and for emotional analysis, it is 76.78% compared to LSTM. This research has a hyperparameter of AB-LaBSE [10]. The experimental results show that the information-based topics proposed by BiLSTM are effective compared to other traditional models for sentiment classification with an accuracy of 85.02% [11].…”

Graph embedding approach to analyze sentiments on cryptocurrency

“…Working with deep learning methods [14], such as bidirectional long short-term memory (Bi-LSTM) [15], is crucial in the sentiment analysis field due to their ability to handle complex patterns in text data. Traditional machine learning techniques are limited in their ability to capture the contextual information and relationships between words in text data.…”