Chinese text classification based on attention mechanism and feature-enhanced fusion neural network

Xie, Jiang; Hou, Yongjin; Wang, Yujing; Wang, Qingyan; Li, Baiwei; Lv, Shiwei; Vorotnitsky, Yury

doi:10.1007/s00607-019-00766-9

Cited by 52 publications

(20 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, a series of model fusion methods have also appeared in the field of deep learning. The methods for model fusion mainly include feature-based methods [7] and scorebased methods [8]. Feature level fusion method is suitable for models of different categories, such as CNN and BiGRU, and can extract word-level features and syntax-level features simultaneously [7].…”

Section: Model Fusion Approaches For Text Classificationmentioning

confidence: 99%

“…The methods for model fusion mainly include feature-based methods [7] and scorebased methods [8]. Feature level fusion method is suitable for models of different categories, such as CNN and BiGRU, and can extract word-level features and syntax-level features simultaneously [7]. While the fusion methods based on the score level are more applicable to similar structure for models, which obtain the final result by voting.…”

Section: Model Fusion Approaches For Text Classificationmentioning

confidence: 99%

See 1 more Smart Citation

Transformer-Based Language Model Fine-Tuning Methods for COVID-19 Fake News Detection

Chen

Gao

et al. 2021

Communications in Computer and Information Science

View full text Add to dashboard Cite

With the pandemic of COVID-19, relevant fake news is spreading all over the sky throughout the social media. Believing in them without discrimination can cause great trouble to people's life. However, universal language models may perform weakly in these fake news detection for lack of large-scale annotated data and sufficient semantic understanding of domain-specific knowledge. While the model trained on corresponding corpora is also mediocre for insufficient learning. In this paper, we propose a novel transformer-based language model fine-tuning approach for these fake news detection. First, the token vocabulary of individual model is expanded for the actual semantics of professional phrases. Second, we adapt the heated-up softmax loss to distinguish the hard-mining samples, which are common for fake news because of the disambiguation of short text. Then, we involve adversarial training to improve the model's robustness. Last, the predicted features extracted by universal language model RoBERTa and domain-specific model CT-BERT are fused by one multiple layer perception to integrate fine-grained and high-level specific representations. Quantitative experimental results evaluated on existing COVID-19 fake news dataset show its superior performances compared to the state-of-the-art methods among various evaluation metrics. Furthermore, the best weighted average F1 score achieves 99.02%.

show abstract

Section: Model Fusion Approaches For Text Classificationmentioning

confidence: 99%

Section: Model Fusion Approaches For Text Classificationmentioning

confidence: 99%

Transformer-Based Language Model Fine-Tuning Methods for COVID-19 Fake News Detection

Chen

Gao

et al. 2021

Communications in Computer and Information Science

View full text Add to dashboard Cite

show abstract

“…Li et al (2020) proposed a sentiment analysis model based on deep learning, lexicon-integrated dual-channel CNN-LSTM family model, which combined CNN and LSTM/BiLSTM branches in parallel, and when tested found to be superior to many baseline methods in experiments on Chinese comment text data sets. Xie et al (2020) proposed the LSTM Chinese text classification algorithm based on attention mechanism and feature enhancement fusion, which not only increased the weight of important text features but also enhanced the difference between them and other text features, significantly improving the recognition ability of Chinese text feature.…”

Section: Application Of Recurrent Neural Network In Chinese Text Clasmentioning

confidence: 99%

Automatic Subject Classification of Public Messages in E-government Affairs

Pan

Yijin

2021

Data and Information Management

View full text Add to dashboard Cite

Public messages on the Internet political inquiry platform rely on manual classification, which has the problems of heavy workload, low efficiency, and high error rate. A Bi-directional long short-term memory (Bi-LSTM) network model based on attention mechanism was proposed in this paper to realize the automatic classification of public messages. Considering the network political inquiry data set provided by the BdRace platform as samples, the Bi-LSTM algorithm is used to strengthen the correlation between the messages before and after the training process, and the semantic attention to important text features is strengthened in combination with the characteristics of attention mechanism. Feature weights are integrated through the full connection layer to carry out classification calculations. The experimental results show that the F1 value of the message classification model proposed here reaches 0.886 and 0.862, respectively, in the data set of long text and short text. Compared with three algorithms of long short-term memory (LSTM), logistic regression, and naive Bayesian, the Bi-LSTM model can achieve better results in the automatic classification of public message subjects.

show abstract

“…The database entities involved in the legal text classification system mainly include node, source, derivation equation, relevant regulations, case logical section, expression node, expression, thesaurus, and thesaurus keywords [20]. Node description provides basic information about a node.…”

Section: Image Segmentation Algorithm Text Classificationmentioning

confidence: 99%

Application of Image Segmentation Algorithm Based on Partial Differential Equation in Legal Case Text Classification

Sun

2021

Advances in Mathematical Physics

View full text Add to dashboard Cite

As a means of regulating people’s code of conduct, law has a close relationship with text, and text data has been growing exponentially. Managing and classifying huge text data have become a huge challenge. The PDES image segmentation algorithm is an effective natural language processing method for text classification management. Based on the study of image segmentation algorithm and legal case text classification theory, an image segmentation model based on partial differential equation is proposed, in which diffusion indirectly acts on level set function through auxiliary function. The software architecture of image segmentation algorithm text classification system is proposed by using computer technology and three-layer architecture model, which can improve the classification ability of text classification algorithm. The validity of pDE image segmentation model is verified by experiments. The experimental results show that the model completes the legal case text classification, the performance of each functional module of the legal case text classification system is good, and the efficiency and quality of the legal case text classification are improved.

show abstract

Chinese text classification based on attention mechanism and feature-enhanced fusion neural network

Cited by 52 publications

References 14 publications

Transformer-Based Language Model Fine-Tuning Methods for COVID-19 Fake News Detection

Transformer-Based Language Model Fine-Tuning Methods for COVID-19 Fake News Detection

Automatic Subject Classification of Public Messages in E-government Affairs

Application of Image Segmentation Algorithm Based on Partial Differential Equation in Legal Case Text Classification

Contact Info

Product

Resources

About