Enhancing Local Feature Extraction with Global Representation for Neural Text Classification

Niu, Guocheng; Xu, Hengru; He, Bolei; Xiao, Xinyan; Wu, Hua; Gao, Sheng

doi:10.18653/v1/d19-1047

Cited by 14 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Qin et al [ 4 ] proposed a feature projection layer to eliminate the redundant information of features and improve the quality of features. G Niu et al [ 14 ] proposed a new Encoder1–Encoder2 structure, where Encoder1 is a global information extractor and Encoder2 is a local information extractor. The global information vectors are merged with the local information vectors for a higher performance.…”

Section: Related Workmentioning

confidence: 99%

Enhancing the Generalization for Text Classification through Fusion of Backward Features

Seng

2023

Sensors

View full text Add to dashboard Cite

Generalization has always been a keyword in deep learning. Pretrained models and domain adaptation technology have received widespread attention in solving the problem of generalization. They are all focused on finding features in data to improve the generalization ability and to prevent overfitting. Although they have achieved good results in various tasks, those models are unstable when classifying a sentence whose label is positive but still contains negative phrases. In this article, we analyzed the attention heat map of the benchmarks and found that previous models pay more attention to the phrase rather than to the semantic information of the whole sentence. Moreover, we proposed a method to scatter the attention away from opposite sentiment words to avoid a one-sided judgment. We designed a two-stream network and stacked the gradient reversal layer and feature projection layer within the auxiliary network. The gradient reversal layer can reverse the gradient of features in the training stage so that the parameters are optimized following the reversed gradient in the backpropagation stage. We utilized an auxiliary network to extract the backward features and then fed them into the main network to merge them with normal features extracted by the main network. We applied this method to the three baselines of TextCNN, BERT, and RoBERTa using sentiment analysis and sarcasm detection datasets. The results show that our method can improve the sentiment analysis datasets by 0.5% and the sarcasm detection datasets by 2.1%.

show abstract

Section: Related Workmentioning

confidence: 99%

Enhancing the Generalization for Text Classification through Fusion of Backward Features

Seng

2023

Sensors

View full text Add to dashboard Cite

show abstract

“…The syntactic dependency tree is highly structured, which inspires us to capture more structural information from parser model. Convolutional Neural Network (CNN) (LeCun et al, 1998) has been proven that retains the capacity of capturing local structure (Niu et al, 2019). Therefore, we adopt a parallel CNN structure query-key CNN (QKCNN) as the structural information extractor (Yang et al, 2018), where QKCNN is composed of query CNN and key CNN.…”

Section: Structural Extractormentioning

confidence: 99%

Unsupervised Domain Adaptation Method with Semantic-Structural Alignment for Dependency Parsing

Lin¹,

Li²,

Li³

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

Unsupervised cross-domain dependency parsing is to accomplish domain adaptation for dependency parsing without using labeled data in target domain. Existing methods are often of the pseudo-annotation type, which generates data through self-annotation of the base model and performing iterative training. However, these methods fail to consider the change of model structure for domain adaptation. In addition, the structural information contained in the text cannot be fully exploited. To remedy these drawbacks, we propose a Semantics-Structure Adaptative Dependency Parser (SSADP), which accomplishes unsupervised cross-domain dependency parsing without relying on pseudo-annotation or data selection. In particular, we design two feature extractors to extract semantic and structural features respectively. For each type of features, a corresponding feature adaptation method is utilized to achieve domain adaptation to align the domain distribution, which effectively enhances the unsupervised crossdomain transfer capability of the model. We validate the effectiveness of our model by conducting experiments on the CODT1 and CTB9 respectively, and the results demonstrate that our model can achieve consistent performance improvement. Besides, we verify the structure transfer ability of the proposed model by introducing Weisfeiler-Lehman Test.

show abstract

“…Global Context Encoder. Considering a complaint text contains several fine-grained clauses, we introduce a global context encoder [17] to capture the global textual context information 𝑐 to help identify the element of clauses in the complaint text 𝑥. We concatenate all clauses in the complaint text 𝑥 𝑐 = 𝑐𝑜𝑛𝑐𝑎𝑡 ( [𝑥 1 , ..., 𝑥 𝑛 ]) and take the embeddings of them as the input of the global encoder, and use a bi-directional GRU [9] to encode the whole text sequence.…”

Section: Our Modelmentioning

confidence: 99%

Fine-Grained Element Identification in Complaint Text of Internet Fraud

Liu

Wang

et al. 2021

Proceedings of the 30th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

Existing system dealing with online complaint provides a final decision without explanations. We propose to analyse the complaint text of internet fraud in a fine-grained manner. Considering the complaint text includes multiple clauses with various functions, we propose to identify the role of each clause and classify them into different types of fraud element. We construct a large labeled dataset originated from a real finance service platform. We build an element identification model on top of BERT and propose additional two modules to utilize the context of complaint text for better element label classification, namely, global context encoder and label refiner. Experimental results show the effectiveness of our model. CCS CONCEPTS• Applied computing → Secure online transactions.

show abstract

Enhancing Local Feature Extraction with Global Representation for Neural Text Classification

Cited by 14 publications

References 25 publications

Enhancing the Generalization for Text Classification through Fusion of Backward Features

Enhancing the Generalization for Text Classification through Fusion of Backward Features

Unsupervised Domain Adaptation Method with Semantic-Structural Alignment for Dependency Parsing

Fine-Grained Element Identification in Complaint Text of Internet Fraud

Contact Info

Product

Resources

About