Domain-Invariant Feature Distillation for Cross-Domain Sentiment Classification

Hu, Mengting; Wu, Yike; Zhao, Shiwan; Guo, Honglei; Cheng, Renhong; Su, Zhong

doi:10.18653/v1/d19-1558

Cited by 24 publications

(19 citation statements)

References 106 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• BERT (Adv): It fine-tunes BERT by BERT (C) with an additional adversarial domain loss proposed in Hu et al (2019).…”

Section: General Experimental Resultsmentioning

confidence: 99%

“…Additionally, models such as BERT do not have the "shared-private" architecture (Liu et al, 2017), frequently used for transfer learning. One can also replace L AD by asking the classifier to predict the flipped domain labels directly (Shu et al, 2018;Hu et al, 2019). Hence, we can instead minimize the flipped domain loss L F D :…”

Section: Learning Domain-invariant Representationsmentioning

confidence: 99%

“…Given a training instance (x k i , y k i ) of the kth domain, we generate a corrupt domain label z i from a corrupted domain distribution Pr(z i ). z i is unrelated to the true domain label k, which may or may not be the same as k. The goal of the domain classifier is to approximate Pr(z i ) instead of always predicting the incorrect domains as in Hu et al (2019). In practice, Pr(z i ) can be defined with each domain uniformly distributed, if the K domain datasets are relatively balanced in size.…”

Section: Learning Domain-invariant Representationsmentioning

confidence: 99%

See 2 more Smart Citations

Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining

Wang¹,

Qiu²,

Huang³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Pre-trained neural language models bring significant improvement for various NLP tasks, by fine-tuning the models on task-specific training sets. During fine-tuning, the parameters are initialized from pre-trained models directly, which ignores how the learning process of similar NLP tasks in different domains is correlated and mutually reinforced. In this paper, we propose an effective learning procedure named Meta Fine-Tuning (MFT), serving as a meta-learner to solve a group of similar NLP tasks for neural language models. Instead of simply multi-task training over all the datasets, MFT only learns from typical instances of various domains to acquire highly transferable knowledge. It further encourages the language model to encode domaininvariant representations by optimizing a series of novel domain corruption loss functions. After MFT, the model can be fine-tuned for each domain with better parameter initialization and higher generalization ability. We implement MFT upon BERT to solve several multi-domain text mining tasks. Experimental results confirm the effectiveness of MFT and its usefulness for few-shot learning. 1

show abstract

“…• BERT (Adv): It fine-tunes BERT by BERT (C) with an additional adversarial domain loss proposed in Hu et al (2019).…”

Section: General Experimental Resultsmentioning

confidence: 99%

Section: Learning Domain-invariant Representationsmentioning

confidence: 99%

Section: Learning Domain-invariant Representationsmentioning

confidence: 99%

See 1 more Smart Citation

Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining

Wang¹,

Qiu²,

Huang³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

show abstract

“…Sentiment analysis has been studied under different settings in the literature (e.g., sentence-level, aspect-level, cross-domain) Chauhan et al, 2019;Hu et al, 2019). For ABSA, the early works have performed feature engineering to produce useful features for the statistical classification models (e.g., SVM) (Wagner et al, 2014).…”

Section: Related Workmentioning

confidence: 99%

Improving Aspect-based Sentiment Analysis with Gated Graph Convolutional Networks and Syntax-based Regulation

Veyseh

Nouri²,

Dernoncourt

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Aspect-based Sentiment Analysis (ABSA) seeks to predict the sentiment polarity of a sentence toward a specific aspect. Recently, it has been shown that dependency trees can be integrated into deep learning models to produce the state-of-the-art performance for ABSA. However, these models tend to compute the hidden/representation vectors without considering the aspect terms and fail to benefit from the overall contextual importance scores of the words that can be obtained from the dependency tree for ABSA. In this work, we propose a novel graph-based deep learning model to overcome these two issues of the prior work on ABSA. In our model, gate vectors are generated from the representation vectors of the aspect terms to customize the hidden vectors of the graph-based models toward the aspect terms. In addition, we propose a mechanism to obtain the importance scores for each word in the sentences based on the dependency trees that are then injected into the model to improve the representation vectors for ABSA. The proposed model achieves the state-of-the-art performance on three benchmark datasets.

show abstract

“…To further improve the performance, methods of exploiting additional information have been proposed (Li et al 2018;Peng et al 2018;He et al 2018;Hu et al 2019;Bahdanau, Cho, and Bengio 2015). Among these works, Peng et al (2018) and He et al (2018) introduced semisupervised learning methods to utilize the target domain information.…”

Section: Introductionmentioning

confidence: 99%

Topic Driven Adaptive Network for Cross-Domain Sentiment Classification

Zhu¹,

Yiqiao²,

Wu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Cross-domain sentiment classification has been a hot spot these years, which aims to learn a reliable classifier using labeled data from the source domain and evaluate it on the target domain. In this vein, most approaches utilized domain adaptation that maps data from different domains into a common feature space. To further improve the model performance, several methods targeted to mine domain-specific information were proposed. However, most of them only utilized a limited part of domain-specific information. In this study, we first develop a method of extracting domain-specific words based on the topic information. Then, we propose a Topic Driven Adaptive Network (TDAN) for cross-domain sentiment classification. The network consists of two subnetworks: semantics attention network and domain-specific word attention network, the structures of which are based on transformers. These sub-networks take different forms of input and their outputs are fused as the feature vector. Experiments validate the effectiveness of our TDAN on sentiment classification across domains.

show abstract

Domain-Invariant Feature Distillation for Cross-Domain Sentiment Classification

Cited by 24 publications

References 106 publications

Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining

Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining

Improving Aspect-based Sentiment Analysis with Gated Graph Convolutional Networks and Syntax-based Regulation

Topic Driven Adaptive Network for Cross-Domain Sentiment Classification

Contact Info

Product

Resources

About