Supervised Contrastive Learning with Term Weighting for Improving Chinese Text Classification

Abstract

With the rapid growth of information retrieval technology, Chinese text classification, which is the basis of information content security, has become a widely discussed topic. In view of the huge difference compared with English, Chinese text task is more complex in semantic information representations. However, most existing Chinese text classification approaches typically regard feature representation and feature selection as the key points, but fail to take into account the learning strategy that adapts to the task. Besides, these approaches compress the Chinese word into a representation vector, without considering the distribution of the term among the categories of interest. In order to improve the effect of Chinese text classification, a unified method, called Supervised Contrastive Learning with Term Weighting (SCL-TW), is proposed in this paper. Supervised contrastive learning makes full use of a large amount of unlabeled data to improve model stability. In SCL-TW, we calculate the score of term weighting to optimize the process of data augmentation of Chinese text. Subsequently, the transformed features are fed into a temporal convolution network to conduct feature representation. Experimental verifications are conducted on two Chinese benchmark datasets. The results demonstrate that SCL-TW outperforms other advanced Chinese text classification approaches by an amazing margin.

ConKgPrompt: Contrastive Sample Method Based on Knowledge-Guided Prompt Learning for Text Classification

“…However, in the case of supervision, the scarcity of data does not aid in enhancing the semantic representation of sentences. Guo et al [27] proposed a supervised word-weighted contrastive learning method that enhances source text data via the calculation of word weight and utilizes an adversarial approach in combination with supervised contrastive learning. However, this method requires a large amount of labeled data, and the methodology used in constructing positive and negative samples cannot guarantee semantic consistency.…”