REKP: Refined External Knowledge into Prompt-Tuning for Few-Shot Text Classification

Abstract

Text classification is a machine learning technique employed to assign a given text to predefined categories, facilitating the automatic analysis and processing of textual data. However, an important problem is that the number of new text categories is growing faster than that of human annotation data, which makes many new categories of text data lack a lot of annotation data. As a result, the conventional deep neural network is forced to over-fit, which damages the application in the real world. As a solution to this problem, academics recommend addressing data scarcity through few-shot learning. One of the efficient methods is prompt-tuning, which transforms the input text into a mask prediction problem featuring [MASK]. By utilizing descriptors, the model maps output words to labels, enabling accurate prediction. Nevertheless, the previous prompt-based adaption approaches often relied on manually produced verbalizers or a single label to represent the entire label vocabulary, which makes the mapping granularity low, resulting in words not being accurately mapped to their label. To address these issues, we propose to enhance the verbalizer and construct the refined external knowledge into a prompt-tuning (REKP) model. We employ the external knowledge bases to increase the mapping space of tagged terms and design three refinement methods to remove noise data. We conduct comprehensive experiments on four benchmark datasets, namely AG’s News, Yahoo, IMDB, and Amazon. The results demonstrate that REKP can outperform the state-of-the-art baselines in terms of Micro-F1 on knowledge-enhanced text classification. In addition, we conduct an ablation study to ascertain the functionality of each module in our model, revealing that the refinement module significantly contributes to enhancing classification accuracy.

Domain-Specific Few-Shot Table Prompt Question Answering via Contrastive Exemplar Selection

“…Then, the one-class support vector machine-based unlabeled data leveraging (OCSVM-UDL) module to establish an OCSVM model in the trained text vector space, selects reasonable pseudo-label data for each category from a large amount of unlabeled data. Dang et al [22] propose to enhance the verbalizer and construct the refined external knowledge into a prompt-tuning (REKP) model. They employ external knowledge bases to increase the mapping space of tagged terms and design three refinement methods to remove noise data.…”

Proceedings of the 4th New Frontiers in Summarization Workshop

“…Recently, with the availability of large-scale CLS datasets (Zhu et al, 2019;Ladhak et al, 2020;Perez-Beltrachini and Lapata, 2021;Wang et al, 2022a;Zheng et al, 2022), many researchers shift the research attention to end-to-end CLS models. According to a comprehensive CLS review (Wang et al, 2022b), the end-to-end models involve multi-task learning (Cao et al, 2020;Bai et al, 2021b;Liang et al, 2022b), knowledge distillation (Nguyen and Luu, 2022), resourceenhanced Jiang et al, 2022) and pre-training Chi et al, 2021) frameworks. However, none of them explore LLMs performance on CLS.…”

“…Previous works have explored to incorporate control * Work done during internship at I 2 R, A*STAR. signals during pre-training (Keskar et al, 2019), task-specific fine-tuning , and prompt tuning (Zhang et al, 2022b). Meanwhile, the advancements in LLMs have unveiled new paradigms.…”