UMP-MG: A Uni-directed Message-Passing Multi-label Generation Model for Hierarchical Text Classification

Abstract

Hierarchical Text Classification (HTC) is a formidable task which involves classifying textual descriptions into a taxonomic hierarchy. Existing methods, however, have difficulty in adequately modeling the hierarchical label structures, because they tend to focus on employing graph embedding methods to encode the hierarchical structure while disregarding the fact that the HTC labels are rooted in a tree structure. This is significant because, unlike a graph, the tree structure inherently has a directive that ordains information flow from one node to another—a critical factor when applying graph embedding to the HTC task. But in the graph structure, message-passing is undirected, which will lead to the imbalance of message transmission between nodes when applied to HTC. To this end, we propose a unidirectional message-passing multi-label generation model for HTC, referred to as UMP-MG. Instead of viewing HTC as a classification problem as previous methods have done, this novel approach conceptualizes it as a sequence generation task, introducing prior hierarchical information during the decoding process. This further enables the blocking of information flow in one direction to ensure that the graph embedding method is better suited for the HTC task and thus resulted in the enhanced tree structure representation. Results obtained through experimentation on both the public WOS dataset and an E-commerce user intent classification dataset demonstrate that our proposed model can achieve superlative results.

Hierarchical Text Classification and Its Foundations: A Review of Current Research

“…The latter can be found in Tables 9 and 10, while the results over the individual splits are reported in Table 11. [149] 0.810 0.533 --HiLAP [99] 0.833 0.601 --HiAGM-TP [97] 0.840 0.634 0.858 0.803 RLHR [167] --0.785 0.792 HCSM [168] 0.858 0.609 0.921 0.807 HiMatch [124] 0.847 0.641 0.862 0.805 HIDDEN [169] 0.793 0.473 --HE-AGCRCNN [170] 0.778 0.513 --HVHMC [171] --0.743 -SASF [126] --0.867 0.811 HTCInfoMax [177] 0.835 0.627 0.856 0.800 PAAM-HiA-T5 [178] 0.872 0.700 0.904 0.816 HPT [136] 0.873 0.695 0.872 0.819 HGCLR [91] 0.865 0.683 0.871 0.812 Seq2Tree [93] 0.869 0.700 0.872 0.825 HBGL [180] 0.872 0.711 0.874 0.820 P-tuning v2 (SPP-tuning) [138] --0.875 0.800 LD-GGNN [186] 0.842 0.641 0.851 0.805 LSE-HiAGM [123] 0.839 0.646 0.860 0.800 Seq2Label [121] 0.874 0.706 0.873 0.819 HTC-CLIP [190] --0.879 0.816 GACaps [191] 0.868 0.698 0.876 0.828 HiDEC [194] 0.855 0.651 --UMP-MG [129] --0.859 0.813 LED [196] 0.883 0.697 0.870 0.813 (HGCLR-based + aug) [198] 0.862 0.679 0.874 0.821 K-HTC [135] --0.873 0.817 HiTIN (BERT) [200] 0.867 0.699 0.872 0.816 HierVerb (few-shot) [137] 0 The standard deviation over the 2 repetitions of 3-fold cross-validation is reported in brackets.…”

“…Additional techniques are often applied to improve label correlation and to better model their cooccurrence [125][126][127][128]. Some authors, such as Ning et al [129], argue that graph-based approaches may lose on the directed nature of the tree hierarchy and thus introduce unidirectional message-passing constraints to improve graph embedding. Other works utilize GCNs to reinforce feature sharing among labels of the same level [130].…”