Chinese short text classification by combining Bert and graph convolutional network

Abstract

Short Text Classification is the fundamental task in the nature language processing. There is a lack of language structure and uneven classification of data samples in short texts, which limit the development of deep learning based short text classification. To address the limitations of text sequences, we propose using a large-scale pre-trained language model Bert to obtain feature information between words and bureaus in the text, Graph Convolutional Network (GCN) with double-layer convolutional network can obtain the dependency relationships between words. We propose to combine Bert with GCN in short Chinese medical texts, where BertGCN outperforms better than other’s methods in classification accuracy.