Using the Past Knowledge to Improve Sentiment Classification

Qin, Qi; Hu, Wenpeng; Liu, Bing

doi:10.18653/v1/2020.findings-emnlp.101

Cited by 11 publications

(11 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Wang et al (2018) used LL for ASC, but improved only the new task and did not deal with CF. Existing CL systems SRK (Lv et al, 2019), KAN (Ke et al, 2020b) and L2PG (Qin et al, 2020) are for document sentiment classification, but not ASC. Ke et al (2020a) also performed transfer in the image domain.…”

Section: Continual Learning Existing Work Has Mainly Focused On Dealing With Catastrophic Forgetting (Cf)mentioning

confidence: 99%

Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks

Zixuan¹,

Xu²,

Liu³

2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

Self Cite

View full text Add to dashboard Cite

This paper studies continual learning (CL) of a sequence of aspect sentiment classification (ASC) tasks. Although some CL techniques have been proposed for document sentiment classification, we are not aware of any CL work on ASC. A CL system that incrementally learns a sequence of ASC tasks should address the following two issues: (1) transfer knowledge learned from previous tasks to the new task to help it learn a better model, and (2) maintain the performance of the models for previous tasks so that they are not forgotten. This paper proposes a novel capsule network based model called B-CL to address these issues. B-CL markedly improves the ASC performance on both the new task and the old tasks via forward and backward knowledge transfer. The effectiveness of B-CL is demonstrated through extensive experiments. 1

show abstract

Section: Continual Learning Existing Work Has Mainly Focused On Dealing With Catastrophic Forgetting (Cf)mentioning

confidence: 99%

Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks

Zixuan¹,

Xu²,

Liu³

2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

Self Cite

View full text Add to dashboard Cite

show abstract

“…L2PG (Qin et al, 2020) uses a neural network but improves only the new task learning for DSC. Wang et al (2018) worked on ASC, but since they improve only the new task learning, they did not deal with CF.…”

Section: Related Workmentioning

confidence: 99%

“…After learning a task, its training data is often discarded (Chen and Liu, 2018). The CL setting is useful when the data privacy is a concern, i.e., the data owners do not want their data used by others (Ke et al, 2020b;Qin et al, 2020;Ke et al, 2021). In such cases, if we want to leverage the knowledge learned in the past to improve the new task learning, CL is appropriate as it shares only the learned model, but not the data.…”

Section: Introductionmentioning

confidence: 99%

CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks

Zixuan¹,

Liu²,

Xu³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper studies continual learning (CL) of a sequence of aspect sentiment classification (ASC) tasks in a particular CL setting called domain incremental learning (DIL). Each task is from a different domain or product. The DIL setting is particularly suited to ASC because in testing the system needs not know the task/domain to which the test data belongs. To our knowledge, this setting has not been studied before for ASC. This paper proposes a novel model called CLASSIC. The key novelty is a contrastive continual learning method that enables both knowledge transfer across tasks and knowledge distillation from old tasks to the new task, which eliminates the need for task ids in testing. Experimental results show the high effectiveness of CLASSIC. 1 * Work was done prior to joining Amazon.

show abstract

“…Some NLP applications have also dealt with CF. For example, CL models have been proposed for sentiment analysis [23,24,37,43], dialogue slot filling [53], language modeling [58,7], language learning [31], sentence embedding [33], machine translation [25], cross-lingual modeling [35], and question answering [12]. A dialogue CL dataset is also reported in [38].…”

Section: Related Workmentioning

confidence: 99%

“…[55] used LL for aspect extraction. [43] and [62] used neural networks for DSC and ASC, respectively. Several papers also studied lifelong topic modeling [5,13].…”

Section: Related Workmentioning

confidence: 99%

Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning

Zixuan

Liu

Ma³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Continual learning (CL) learns a sequence of tasks incrementally with the goal of achieving two main objectives: overcoming catastrophic forgetting (CF) and encouraging knowledge transfer (KT) across tasks. However, most existing techniques focus only on overcoming CF and have no mechanism to encourage KT, and thus do not do well in KT. Although several papers have tried to deal with both CF and KT, our experiments show that they suffer from serious CF when the tasks do not have much shared knowledge. Another observation is that most current CL methods do not use pre-trained models, but it has been shown that such models can significantly improve the end task performance. For example, in natural language processing, fine-tuning a BERT-like pre-trained language model is one of the most effective approaches. However, for CL, this approach suffers from serious CF. An interesting question is how to make the best use of pre-trained models for CL. This paper proposes a novel model called CTR to solve these problems. Our experimental results demonstrate the effectiveness of CTR. 2 * Work was done prior to joining Amazon. 2 The code of CTR can be found at https://github.com/ZixuanKe/PyContinual 35th Conference on Neural Information Processing Systems (NeurIPS 2021),

show abstract

Using the Past Knowledge to Improve Sentiment Classification

Cited by 11 publications

References 19 publications

Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks

Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks

CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks

Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning

Contact Info

Product

Resources

About