Research on sentiment classification for netizens based on the BERT-BiLSTM-TextCNN model

Abstract

Sentiment analysis of netizens’ comments can accurately grasp the psychology of netizens and reduce the risks brought by online public opinion. However, there is currently no effective method to solve the problems of short text, open word range, and sometimes reversed word order in comments. To better solve the above problems, this article proposes a hybrid model of sentiment classification, which is based on bidirectional encoder representations from transformers (BERT), bidirectional long short-term memory (BiLSTM) and a text convolution neural network (TextCNN) (BERT-BiLSTM-TextCNN). The experimental results show that (1) the hybrid model proposed in this article can better combine the advantages of BiLSTM and TextCNN; it not only captures local correlation while retaining context information but also has high accuracy and stability. (2) The BERT-BiLSTM-TextCNN model can extract important emotional information more flexibly in text and achieve multiclass classification tasks of emotions more accurately. The innovations of this study are as follows: (1) the use of BERT to generate word vectors has the advantages of more prior information and a full combination of contextual semantics; (2) the BiLSTM model, as a bidirectional context mechanism model, can obtain contextual information well; and (3) the TextCNN model can obtain important features well in the problem of text classification, and the combined effect of the three modules can significantly improve the accuracy of emotional multilabel classification.

PLM_Sol: predicting protein solubility by benchmarking multiple protein language models with the updatedEscherichia coliprotein solubility dataset

“…In this work, three PLMs have been tested, including ProteinBERT (34), ProtTrans (37) and ESM2 (Table 1). Additionally, three classification architectures have been employed, namely MLP, LA (45) and biLSTM_TextCNN (46). A total of 13 models were developed and trained (Methods).…”

“…The second is the LA architecture, as described in a previous study (45), which shows excellent performance in protein localization classification. Thirdly, the biLSTM_TextCNN architecture, known for its effectiveness in sentiment classification (46), utilizes a bidirectional LSTM (biLSTM) to convert the information from the encoder into corresponding matrices. Following the classification process, a Text-attentional CNN (TextCNN) is applied, featuring three parallel one-dimensional convolution (Conv-1D) layers for crucial feature extraction.…”

“…For model design, we trained a series of architectures by integrating multiple PLMs with diverse classifiers. Three pretrained PLMs, namely proteinBERT, ESM2 and ProtTrans were employed alongside different classification layers, comprising MLP, Light Attention (LA) (45) and the biLSTM_TextCNN (46). Subsequently, we systematically evaluated their performance on an independent test set.…”

Integrating BERT Embeddings and BiLSTM for Emotion Analysis of Dialogue

“…In order to validate the proposed model, some baselines are as follows. BERT-BiLSTM-CNN [ 27 ]: It combines the advantages of BERT embedding, BiLSTM, and TextCNN to capture local correlation and retain context information. BERT-BiGRU-CNN: This baseline replaces BiLSTM in BERT-BiLSTM-CNN with BiGRU, which has the simpler structure and calculation than BiLSTM.…”

“…In sentiment classification, the BiLSTM, BiGRU, and CNN model are integrated and proposed for sentiment classification [ 5 ]. A hybrid model of sentiment classification is proposed, which is based on BERT, BiLSTM, and a text convolution neural network [ 27 ]. In the legal area, a shallow network with one BiLSTM layer and one attention layer is used to perform Portuguese legal text classification [ 28 ].…”