Contrastive Learning with Heterogeneous Graph Attention Networks on Short Text Classification

Abstract

Graph neural networks (GNNs) have attracted extensive interest in text classification tasks due to their expected superior performance in representation learning. However, most existing studies adopted the same semi-supervised learning setting as the vanilla Graph Convolution Network (GCN), which requires a large amount of labelled data during training and thus is less robust when dealing with large-scale graph data with fewer labels. Additionally, graph structure information is normally captured by direct information aggregation via network schema and is highly dependent on correct adjacency information. Therefore, any missing adjacency knowledge may hinder the performance. Addressing these problems, this paper thus proposes a novel method to learn a graph structure, NC-HGAT, by expanding a state-of-the-art self-supervised heterogeneous graph neural network model (HGAT) with simple neighbour contrastive learning. The new NC-HGAT considers the graph structure information from heterogeneous graphs with multilayer perceptrons (MLPs) and delivers consistent results, despite the corrupted neighbouring connections. Extensive experiments have been implemented on four benchmark short-text datasets. The results demonstrate that our proposed model NC-HGAT significantly outperforms state-of-the-art methods on three datasets and achieves competitive performance on the remaining dataset.

Improved Graph Contrastive Learning for Short Text Classification

“…Label information is propagated through message passing on the built graph. Inspired by the success of CL in unsupervised representation learning, recent studies (Su et al 2022) explore the potential of CL based on GNNs to leverage the self-supervised signals present in the unlabeled data, aiding in extracting useful features. However, their effectiveness is heavily relies on the generated contrastive views, and the way the views are generated can easily lead to incorrect self-supervised signals that misguide model learning.…”

“…Therefore, short text classification (STC), as a highly challenging task that attracts tremendous attention from researchers, has a wide range of practical applications, such as news classification (Chen et al 2019), sentiment analysis (Yao, Mao, and Luo 2019), and social media analysis (Liu et al 2021(Liu et al , 2023a. Recently, some studies (Su et al 2022) have attempted to integrate graph neural networks (GNNs) with contrastive learning (CL) for solving STC tasks, with promising results. In these approaches, a corpus-level graph is constructed, incorporating latent topics, words, or entities as nodes.…”

Is Unimodal Bias Always Bad for Visual Question Answering? A Medical Domain Study with Dynamic Attention

“…Additionally, as there is a limited amount of training data available in medical VQA, we can apply graph generative methods [32], to enhance the generalisation ability of models. • Graph representation learning methods can be introduced to the question embeddings, such as heterogeneous graph neural networks for different words [33], [34], [35]. • External knowledge, such as knowledge graphs, can be considered [36], [37], so that the model can understand questions and implement the inference, by connecting questions to knowledge graphs.…”