Few-shot text classification by leveraging bi-directional attention and cross-class knowledge

Pang, Ning; Zhao, Xiang; Wang, Wei; Xiao, Weidong; Guo, Deke

doi:10.1007/s11432-020-3055-1

Cited by 20 publications

(7 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Xu and Xiang [57] proposed a multiperspective aggregation-based graph neural network that observes through eyes (support and query instance) and speaks by mouth (pair) for few-shot text classifcation. Pang et al [58] proposed an adapted bidirectional attention mechanism to exploit the interaction between query and support instances in metric learning to better describe text classifcation features. Te model we constructed was derived from the prototype network.…”

Section: Related Workmentioning

confidence: 99%

CNFRD: A Few-Shot Rumor Detection Framework via Capsule Network for COVID-19

Chen

et al. 2023

International Journal of Intelligent Systems

View full text Add to dashboard Cite

In recent years, COVID-19 has become the hottest topic. Various issues, such as epidemic transmission routes and preventive measures, have “occupied” several online social media platforms. Many rumors about COVID-19 have also arisen, causing public anxiety and seriously affecting normal social order. Identifying a rumor at its very inception is crucial to reducing the potential harm of its evolution to society as a whole. However, epidemic rumors provide limited signal features in the early stage. In order to identify rumors with data sparsity, we propose a few-shot learning rumor detection model based on capsule networks (CNFRD), utilizing the metric learning framework and the capsule network to detect the rumors posted during unexpected epidemic events. Specifically, we constructively use the capsule network neural layer to summarize the historical rumor data and obtain the generalized class representation based on the historical rumor data samples. Besides, we calculate the distance between the epidemic rumor sample and the historical rumor class-wise representation according to the metric module. Finally, epidemic rumors are discriminated against according to the nearest neighbor principle. The experimental results prove that the proposed method can achieve higher accuracy with fewer epidemic rumor samples. This approach provided 88.92% accuracy on the Chinese rumor dataset and 87.07% accuracy on the English rumor dataset, which improved by 7% to 23% over existing approaches. Therefore, the CNFRD model can identify epidemic rumors in COVID-19 as early as possible and effectively improve the performance of rumor detection.

show abstract

Section: Related Workmentioning

confidence: 99%

CNFRD: A Few-Shot Rumor Detection Framework via Capsule Network for COVID-19

Chen

et al. 2023

International Journal of Intelligent Systems

View full text Add to dashboard Cite

show abstract

“…Bao et al [3] trained a meta-learning framework to map feature codes into attention scores for measuring the vocabulary representations of words, and adopted the ridge regression classifier to predict the results after seeing only a few training examples. Pang et al [20] used a bi-directional attention mechanism to encode the classification features. Despite recent progress, the generalization abilities of existing models are limited, and metalearning application scenarios are in urgent need of improvement.…”

Section: Meta-learning For Text Classificationmentioning

confidence: 99%

Few-Shot Text Classification with Global–Local Feature Information

Wang

Cheng

et al. 2022

Sensors

View full text Add to dashboard Cite

Meta-learning frameworks have been proposed to generalize machine learning models for domain adaptation without sufficient label data in computer vision. However, text classification with meta-learning is less investigated. In this paper, we propose SumFS to find global top-ranked sentences by extractive summary and improve the local vocabulary category features. The SumFS consists of three modules: (1) an unsupervised text summarizer that removes redundant information; (2) a weighting generator that associates feature words with attention scores to weight the lexical representations of words; (3) a regular meta-learning framework that trains with limited labeled data using a ridge regression classifier. In addition, a marine news dataset was established with limited label data. The performance of the algorithm was tested on THUCnews, Fudan, and marine news datasets. Experiments show that the SumFS can maintain or even improve accuracy while reducing input features. Moreover, the training time of each epoch is reduced by more than 50%.

show abstract

“…Heterogeneous model reuse takes advantage of the teacher from a related task, which relieves the burden of data storage so as to decrease the risk of privacy leaking [35], [54]. Meta-learning has also been utilized to transfer knowledge across different label spaces, e.g., fewshot learning [55], [56], [57], [58], [59], [60]. These approaches usually require special training strategies of the teacher.…”

Section: Related Workmentioning

confidence: 99%

Generalized Knowledge Distillation via Relationship Matching

Ye¹,

Lu²,

Zhan³

2022

Preprint

View full text Add to dashboard Cite

The knowledge of a well-trained deep neural network (a.k.a. the "teacher") is valuable for learning similar tasks. Knowledge distillation extracts knowledge from the teacher and integrates it with the target model (a.k.a. the "student"), which expands the student's knowledge and improves its learning efficacy. Instead of enforcing the teacher to work on the same task as the student, we borrow the knowledge from a teacher trained from a general label space -in this "Generalized Knowledge Distillation (GKD)", the classes of the teacher and the student may be the same, completely different, or partially overlapped. We claim that the comparison ability between instances acts as an essential factor threading knowledge across tasks, and propose the RElationship FacIlitated Local cLassifiEr Distillation (REFILLED) approach, which decouples the GKD flow of the embedding and the top-layer classifier. In particular, different from reconciling the instance-label confidence between models, REFILLED requires the teacher to reweight the hard tuples pushed forward by the student and then matches the similarity comparison levels between instances. An embedding-induced classifier based on the teacher model supervises the student's classification confidence and adaptively emphasizes the most related supervision from the teacher. REFILLED demonstrates strong discriminative ability when the classes of the teacher vary from the same to a fully non-overlapped set w.r.t. the student. It also achieves state-of-the-art performance on standard knowledge distillation, one-step incremental learning, and few-shot learning tasks.

show abstract

Few-shot text classification by leveraging bi-directional attention and cross-class knowledge

Cited by 20 publications

References 17 publications

CNFRD: A Few-Shot Rumor Detection Framework via Capsule Network for COVID-19

CNFRD: A Few-Shot Rumor Detection Framework via Capsule Network for COVID-19

Few-Shot Text Classification with Global–Local Feature Information

Generalized Knowledge Distillation via Relationship Matching

Contact Info

Product

Resources

About