Transformer Based Memory Network for Sentiment Analysis of Web Comments

Jiang, Ming; Wu, Junlei; Shi, Xiangrong; Zhang, Min

doi:10.1109/access.2019.2957192

Cited by 29 publications

(18 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For this reason, bidirectional GRU is adopted in the FGAOM to learn long term dependencies from accumulated features in the previous layer in both forward and backward directions. Thus, for feature at time t, the hidden states of forward and backward GRU are computed as shown in equations (8)(9)(10).…”

Section: ) Bi-grumentioning

confidence: 99%

“…MHA mechanism and convolution operation are employed to further analyze the relevance of aspect and context words that passes through interactive pooling layer. [8]: This model applies the transformer memory network to learn hidden contextual patterns in complex web comments via global attention and local attention mechanisms for constructing fine-grained semantic representation.…”

Section: A Compared Studiesmentioning

confidence: 99%

“…In inspiration of this human behavior, researchers widely applied attention mechanism with neural network architectures for many AOM applications. For example, Han et al [7] introduced an attention mechanism with bidirectional GRU (Bi-GRU) for aspect sentiment classification in drug review, also Jiang et al [8] employed global and local selfattention (SA) mechanism to learn textual web comment embedded semantics.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep Learning Model for Fine-Grained Aspect-Based Opinion Mining

et al. 2020

View full text Add to dashboard Cite

Despite the great manufactures' efforts to achieve customer satisfaction and improve their performance, social media opinion mining is still on the fly a big challenge. Current opinion mining requires sophisticated feature engineering and syntactic word embedding without considering semantic interaction between aspect term and opinionated features, which degrade the performance of most of opinion mining tasks, especially those that are designed for smart manufacturing. Research on intelligent aspect level opinion mining (AOM) follows the fast proliferation of user-generated data through social media for industrial manufacturing purposes. Google's pre-trained language model, Bidirectional Encoder Representations from Transformers (BERT) widely overcomes existing methods in eleven natural language processing (NLP) tasks, which makes it the standard way for semantic text representation. In this paper, we introduce a novel deep learning model for fine-grained aspect-based opinion mining, named as FGAOM. First, we train the BERT model on three specific domain corpora for domain adaption, then use adjusted BERT as embedding layer for concurrent extraction of local and global context features. Then, we propose Multi-head Self-Attention (MSHA) to effectively fuse internal semantic text representation and take advantage of convolutional layers to model aspect term interaction with surrounding sentiment features. Finally, the performance of the proposed model is evaluated via extensive experiments on three public datasets. Results show that performance of the proposed model outperforms performances of recent the-of-the-art models.

show abstract

Section: ) Bi-grumentioning

confidence: 99%

Section: A Compared Studiesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning Model for Fine-Grained Aspect-Based Opinion Mining

et al. 2020

View full text Add to dashboard Cite

show abstract

“…,where e d is the dimension of vector. In T-MGAN design, Glove [40] word embeddings are alternatives for the embedding layer. We obtain the specific word vector through lookup embedding…”

Section: B Input Word Embedding Layermentioning

confidence: 99%

“…In the experiment, we set the initial word embeddings to 300-dimension Glove vectors [40] for all datasets. And for the words that out of vocabulary, we randomly sample their embeddings from the uniform distribution ( 0.01,0.01) U  .…”

Section: B Experiments Settingmentioning

confidence: 99%

Transformer Based Multi-Grained Attention Network for Aspect-Based Sentiment Analysis

et al. 2020

View full text Add to dashboard Cite

Aspect-based sentiment analysis aims to predict sentiment polarity for every aspect in a sentence review. Most existing approaches are based on the sequence models, which may superimpose the emotional semantics of different tendencies and lack syntactic structure information. And most models adopt coarse-grained attention mechanism which still face the issues of weakness interaction between aspect and context. In this paper, we propose a transformer based multi-grained attention network (T-MGAN), which utilizes the Transformer module to learn the word-level representations of aspects and context respectively, and further utilizes the Tree Transformer module to obtain the phrase-level representations of contexts. It is capable of extracting the syntactic structure features and syntax information of aspect and context. In addition, we adopt dual-pooling method and multi-grained attention network to extract high quality aspect-context interactive representations. We evaluate the proposed model on three datasets and prove the effectiveness of the proposed model. INDEX TERMS Aspect-based sentiment analysis, transformer, tree transformer, attention mechanism, nature language processing.

show abstract

An improved transformer‐based model for detecting phishing, spam and ham emails: A large language model approach

Jamal,

Wimmer,

Sarker

2024

Security and Privacy

View full text Add to dashboard Cite

Phishing and spam have been a cybersecurity threat with the majority of breaches resulting from these types of social engineering attacks. Therefore, detection has been a long‐standing challenge for both academic and industry researcher. New and innovative approaches are required to keep up with the growing sophistication of threat actors. One such illumination which has vast potential are large language models (LLM). LLM emerged and already demonstrated their potential to transform society and provide new and innovative approaches to solve well‐established challenges. Phishing and spam have caused financial hardships and lost time and resources to email users all over the world and frequently serve as an entry point for ransomware threat actors. While detection approaches exist, especially heuristic‐based approaches, LLMs offer the potential to venture into a new unexplored area for understanding and solving this challenge. LLMs have rapidly altered the landscape from business, consumers, and throughout academia and demonstrate transformational potential to profoundly impact the society. Based on this, applying these new and innovative approaches to email detection is a rational next step in academic research. In this work, we present IPSDM, an improved phishing spam detection model based on fine‐tuning the BERT family of models to specifically detect phishing and spam emails. We demonstrate our fine‐tuned version, IPSDM, is able to better classify emails in both unbalanced and balanced datasets. Moreover, IPSDM consistently outperforms the baseline models in terms of classification accuracy, precision, recall, and F1‐score, while concurrently mitigating overfitting concerns.

show abstract

Transformer Based Memory Network for Sentiment Analysis of Web Comments

Cited by 29 publications

References 27 publications

Deep Learning Model for Fine-Grained Aspect-Based Opinion Mining

Deep Learning Model for Fine-Grained Aspect-Based Opinion Mining

Transformer Based Multi-Grained Attention Network for Aspect-Based Sentiment Analysis

An improved transformer‐based model for detecting phishing, spam and ham emails: A large language model approach

Contact Info

Product

Resources

About