Hierarchy-Aware Global Model for Hierarchical Text Classification

Abstract

Hierarchical text classification is an essential yet challenging subtask of multi-label text classification with a taxonomic hierarchy. Existing methods have difficulties in modeling the hierarchical label structure in a global view. Furthermore, they cannot make full use of the mutual interactions between the text feature space and the label space. In this paper, we formulate the hierarchy as a directed graph and introduce hierarchy-aware structure encoders for modeling label dependencies. Based on the hierarchy encoder, we propose a novel end-to-end hierarchy-aware global model (Hi-AGM) with two variants. A multi-label attention variant (HiAGM-LA) learns hierarchyaware label embeddings through the hierarchy encoder and conducts inductive fusion of labelaware text features. A text feature propagation model (HiAGM-TP) is proposed as the deductive variant that directly feeds text features into hierarchy encoders. Compared with previous works, both HiAGM-LA and HiAGM-TP achieve significant and consistent improvements on three benchmark datasets.

HTCInfoMax: A Global Model for Hierarchical Text Classification via Information Maximization

“…The major part of HTCInfoMax is the "Information Maximization" part shown in the dashed box which has two new modules: text-label mutual information maximization and label prior matching, which will be introduced in the following sections. We keep the remaining part such as text encoder, structure encoder and the predictor be the same as in HiAGM-LA (Zhou et al, 2020).…”

“…And the loss from the predictor is the traditional binary cross-entropy loss L c (Zhou et al, 2020).…”

“…Following HiAGM (Zhou et al, 2020), we use RCV1-V2 (Lewis et al, 2004) and Web of Science (WOS) (Kowsari et al, 2017) Standard evaluation metrics including Micro-F1 (Mi-F1) and Macro-F1 (Ma-F1) score are employed to evaluate our model. In label imbalance situation, Ma-F1 can better evaluate model's performance in the perspective of not focusing on frequent labels in a certain degree.…”

“…The earlier global approaches ignore the hierarchical structure of labels and assume there is no dependency among labels which leads to flat models such as (Johnson and Zhang, 2015). Later on, more and more works try to make use of the label taxonomic hierarchy to improve the performance by employing different strategies such as recursively regularized Graph-CNN (Peng et al, 2018), reinforcement learning (Mao et al, 2019), attentional capsule network (Peng et al, 2019), meta-learning (Wu et al, 2019) and structure encoder (Zhou et al, 2020). Many attention-based models are also proposed to learn more refined text features for text classification tasks such as (You et al, 2019;Deng et al, 2020).…”

“…Many attention-based models are also proposed to learn more refined text features for text classification tasks such as (You et al, 2019;Deng et al, 2020). Among these methods, HiAGM proposed by Zhou et al (2020) is the state-of-the-art model for HTC which designs a structure encoder that integrates the label prior hierarchy knowledge to learn label representations, and then proposes a model HiAGM with two variants (one is HiAGM-LA, the other is HiAGM-TP) based on the structure encoder to capture the interactions between text features and label representations. However, there are some limitations of HiAGM.…”