Sentiment Analysis About Investors and Consumers in Energy Market Based on BERT-BiLSTM

Abstract

With the rapid development of social media, the number of online comments has exploded, and more and more people are willing to express their attitudes and feelings on the Internet. Under the influence of a series of major events all over the world, production order is facing a serious challenge, which severely impact on energy market. In 2020, a large number of investors' and consumers' comments related to social events began to appear on the Internet from China. However, the style and quality of online comments vary greatly, making it difficult to accurately extract users' views and tendencies. Based on the investors' and consumers' statements published on Chinese Internet, in this paper, we use statistical methods to classify the sentiment orientation of the Netizens firstly, and then use Bidirectional Encoder Representations from Transformer-Bidirectional Long Short Term Memory (BERT-BiLSTM) which is the combination forecasting method of Bidirectional Encoder Representations from Transformer (BERT) and Bidirectional Long Short Term Memory (BiLSTM), to model and forecast the sentiment orientation of users' statements, as well as being compared with its based models, BERT and BiLSTM. Among them, the accuracy and recall，which represent the predictive abilities on the overall samples and on the focus(the samples of Label 1) respectively, of BERT-BiLSTM model are 0.8620 and 0.7078 respectively, which are superior to those of BERT model (0.8559 and 0.5576) and LSTM model (0.7775 and 0.0747). The research results can accurately predict the sentiment orientation of Internet users during the social events so as to provide technical support for grasping the energy market trend. INDEX TERMS sentiment analysis, combination model, BERT-BiLSTM, energy market, investors and consumers.

Tuned bidirectional encoder representations from transformers for fake news detection

“…Attention mechanism is obtained by adopting encoder-decoder architectures to transformers architectures with the aim to find the "summary" of the data with encoder and then the decoder translate them. In this sense, the transformers try to find mindfulness and find a way of looking at the relationships between words in a particular sentence [17], [18]. Currently, BERT is one of the most powerful representations of context and word.…”

Aspect-Level Sentiment Analysis Using CNN Over BERT-GCN

“…In this study, we employed the BiLSTM model [9] over BERT embeddings for contextualized word representation. BiLSTM can learn the context information and latent meaning of words by reading the input sentence in two directions [3]. Contextualized representations were created according to the following steps:…”

“…Unlike the basic grammar models that mainly depend on the statistical characteristics, BiLSTM is competent in learning context information by encoding the sentence according to both directions. Therefore, BiLSTM can capture the real meaning hidden between words from their context [3]. GCNs can effectively learn graph representations and have obtained satisfactory results in various applications and tasks, particularly classification tasks [25].…”

Weibo Text Sentiment Analysis Based on BERT and Deep Learning

“…Because the symbol is different from other words in the text, this symbol independent of the input text does not carry obvious semantic information, so it will output the result more objectively and impartially. The output can effectively integrate the semantic information carried by each word in the input text and then better represent the text [27]. The model structure of BERT is shown in Figure 1.…”

“…BERT+BiLSTM+Att [27]: The words are dynamically converted into word vectors through BERT, so that the converted word vectors are closer to the context. The generated word vectors are used to extract the sentiment features through the BiLSTM network, and the extracted feature vectors are given different attention weights through the attention mechanism.…”