Feature Fusion Text Classification Model Combining CNN and BiGRU with Multi-Attention Mechanism

Zhang, Jing-Ren; Fang-ai, Liu; Xu, Wei; Yu, Hui

doi:10.3390/fi11110237

Cited by 34 publications

(17 citation statements)

References 26 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the continuous development of deep learning technologies, several new models have been introduced. In particular, the bidirectional gated recurrent unit (BiGRU), developed for a purpose similar to that of our models, is being introduced into text classification technology [28]. Similar to BiLSTM, BiGRU extends GRU, and it shows better performance than BiLSTM.…”

Section: Discussionmentioning

confidence: 99%

Efficient Deep Learning Models for DGA Domain Detection

Namgung

Son

Moon

2021

Security and Communication Networks

View full text Add to dashboard Cite

In recent years, cyberattacks using command and control (C&C) servers have significantly increased. To hide their C&C servers, attackers often use a domain generation algorithm (DGA), which automatically generates domain names for the C&C servers. Accordingly, extensive research on DGA domain detection has been conducted. However, existing methods cannot accurately detect continuously generated DGA domains and can easily be evaded by an attacker. Recently, long short-term memory- (LSTM-) based deep learning models have been introduced to detect DGA domains in real time using only domain names without feature extraction or additional information. In this paper, we propose an efficient DGA domain detection method based on bidirectional LSTM (BiLSTM), which learns bidirectional information as opposed to unidirectional information learned by LSTM. We further maximize the detection performance with a convolutional neural network (CNN) + BiLSTM ensemble model using Attention mechanism, which allows the model to learn both local and global information in a domain sequence. Experimental results show that existing CNN and LSTM models achieved F1-scores of 0.9384 and 0.9597, respectively, while the proposed BiLSTM and ensemble models achieved higher F1-scores of 0.9618 and 0.9666, respectively. In addition, the ensemble model achieved the best performance for most DGA domain classes, enabling more accurate DGA domain detection than existing models.

show abstract

Section: Discussionmentioning

confidence: 99%

Efficient Deep Learning Models for DGA Domain Detection

Namgung

Son

Moon

2021

Security and Communication Networks

View full text Add to dashboard Cite

show abstract

“…GRU networks can only manage sequences from front to back, which leads to information loss. Therefore, bidirectional GRU was widely used in many NLP tasks, including SA, text classification, named entity recognition, and question answering, e.g., [19][20][21][22] to process data in both directions and provide complete context information.…”

Section: Gru Networkmentioning

confidence: 99%

Comparing word embedding models for Arabic aspect category detection using a deep learning-based approach

Bensoltane

Zaki

2021

E3S Web Conf.

View full text Add to dashboard Cite

Aspect category detection (ACD) is a task of aspect-based sentiment analysis (ABSA) that aims to identify the discussed category in a given review or sentence from a predefined list of categories. ABSA tasks were widely studied in English; however, studies in other low-resource languages such as Arabic are still limited. Moreover, most of the existing Arabic ABSA work is based on rule-based or feature-based machine learning models, which require a tedious task of feature-engineering and the use of external resources like lexicons. Therefore, the aim of this paper is to overcome these shortcomings by handling the ACD task using a deep learning method based on a bidirectional gated recurrent unit model. Additionally, we examine the impact of using different vector representation models on the performance of the proposed model. The experimental results show that our model outperforms the baseline and related work models significantly by achieving an enhanced F1-score of more than 7%.

show abstract

“…Sanda et al [30] conducted a comparative analysis of text feature representation models and verified the impact of text feature representation on the classification effect. In addition, multiple network models were combined [31][32][33][34] to apply feature fusion to text classification tasks with good results. Jang et al [35] proposed a hybrid model of BiLSTM and CNN based on the attention mechanism, which captures the correlation of adjacent words to improve the classification accuracy.…”

Section: Introductionmentioning

confidence: 99%

Research on Long Text Classification Model Based on Multi-Feature Weighted Fusion

Yang

Zhou

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

Text classification in the long-text domain has become a development challenge due to the significant increase in text data, complexity enhancement, and feature extraction of long texts in various domains of the Internet. A long text classification model based on multi-feature weighted fusion is proposed for the problems of contextual semantic relations, long-distance global relations, and multi-sense words in long text classification tasks. The BERT model is used to obtain feature representations containing global semantic and contextual feature information of text, convolutional neural networks to obtain features at different levels and combine attention mechanisms to obtain weighted local features, fuse global contextual features with weighted local features, and obtain classification results by equal-length convolutional pooling. The experimental results show that the proposed model outperforms other models in terms of accuracy, precision, recall, F1 value, etc., under the same data set conditions compared with traditional deep learning classification models, and it can be seen that the model has more obvious advantages in long text classification.

show abstract

Feature Fusion Text Classification Model Combining CNN and BiGRU with Multi-Attention Mechanism

Cited by 34 publications

References 26 publications

Efficient Deep Learning Models for DGA Domain Detection

Efficient Deep Learning Models for DGA Domain Detection

Comparing word embedding models for Arabic aspect category detection using a deep learning-based approach

Research on Long Text Classification Model Based on Multi-Feature Weighted Fusion

Contact Info

Product

Resources

About