Prompt-Based Label-Aware Framework for Few-Shot Multi-Label Text Classification

Abstract

Prompt-based learning has demonstrated remarkable success in few-shot text classification, outperforming the traditional fine-tuning approach. This method transforms a text input into a masked language modeling prompt using a template, queries a fine-tuned language model to fill in the mask, and then uses a verbalizer to map the model's output to a predicted class. Previous prompt-based text classification approaches were primarily designed for multi-class classification, taking advantage of the fact that the classes are mutually exclusive and one example belongs to only one class. However, these assumptions do not hold in the context of multi-label text classification, where labels often exhibit correlations with each other. Therefore, we propose a Prompt-based Label-Aware framework for Multi-Label text classification (PLAML) that addresses the challenges. Specifically, PLAML enhances prompt-based learning with three proposed techniques to improve the overall performance for multi-label classification. The techniques include (i) a token weighting algorithm that considers the correlations between labels, (ii) a template for augmenting training samples, making the training process label-aware, and (iii) a dynamic threshold mechanism, refining the prediction condition of each label. Extensive experiments on few-shot text classification across multiple datasets with various languages show that our PLAML outperforms other baseline methods. We also analyzed the effect of each proposed technique to better understand how it is suitable for the multi-label setting. INDEX TERMSFew-shot learning, multi-label classification, natural language processing, prompt-based learning, text classification, verbalizer. PIYAWAT LERTVITTAYAKUMJORN received the Ph.D. degree from the Department of Computing, Imperial College London, U.K., in 2022. Currently, he is a full-time Research Scientist with Google, USA. His research interests include natural language processing, explainable AI, and human-AI collaboration. PEERAPON VATEEKUL (Member, IEEE) received the Ph.D. degree from the Department of Electrical and Computer Engineering, University of Miami (UM), Coral Gables, FL, USA, in 2012. Currently, he is an Associate Professor with the Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Thailand. His research interests include machine learning, data mining, deep learning, text mining, big data analytics, natural language processing, and applied deep learning techniques in various domains, such as healthcare, geoinformatics, hydrometeorology, and energy trading.