Correlation Networks for Extreme Multi-label Text Classification

Xun, Guangxu; Jha, Kishlay; Sun, Jianhui; Zhang, Aidong

doi:10.1145/3394486.3403151

Cited by 35 publications

(14 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Star-Transformer sparsifies the fully connected attention in the Transformer to a star-shaped structure. • BERTXML (Xun et al, 2020) Evaluation Metrics Two widely used metrics, precision at top k (P @k) and Normalized Discounted Cumulative Gains at top k (nDCG@k), are used to evaluate the model performance 4 .…”

Section: Methodsmentioning

confidence: 99%

Beyond Text: Incorporating Metadata and Label Structure for Multi-Label Document Classification using Heterogeneous Graphs

Ye¹,

Zhang²,

He³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Multi-label document classification, associating one document instance with a set of relevant labels, is attracting more and more research attention. Existing methods explore the incorporation of information beyond text, such as document metadata or label structure. These approaches however either simply utilize the semantic information of metadata or employ the predefined parent-child label hierarchy, ignoring the heterogeneous graphical structures of metadata and labels, which we believe are crucial for accurate multi-label document classification. Therefore, in this paper, we propose a novel neural network based approach for multi-label document classification, in which two heterogeneous graphs are constructed and learned using heterogeneous graph transformers. One is metadata heterogeneous graph, which models various types of metadata and their topological relations. The other is label heterogeneous graph, which is constructed based on both the labels' hierarchy and their statistical dependencies. Experimental results on two benchmark datasets show the proposed approach outperforms several stateof-the-art baselines.

show abstract

Section: Methodsmentioning

confidence: 99%

Beyond Text: Incorporating Metadata and Label Structure for Multi-Label Document Classification using Heterogeneous Graphs

Ye¹,

Zhang²,

He³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

show abstract

“…When the label space is large (e.g., 10K), one [CLS] token (e.g., a 100-dimensional vector) may not be informative enough to predict the relevant labels. Therefore, following [58], we put multiple [CLS] tokens [CLS 1 ], ..., [CLS 𝐶 ] in the input. To summarize, given a document 𝑑, the layer input 𝑯 is…”

Section: Transformer Layersmentioning

confidence: 99%

“…Considering the sparsity of labels, a short-ranked list of potentially relevant labels for each testing document is commonly used to represent classification quality. Following previous studies on extreme multi-label text classification [27,58,63], we adopt two rank-based metrics: the precision at top 𝑘 (P@𝑘) and the normalized Discounted Cumulative Gain at top 𝑘 (NDCG@𝑘), where 𝑘 = 1, 3, 5. For a document 𝑑, let 𝒚 𝑑 ∈ {0, 1} | L | be its ground truth label vector and rank(𝑖) be the index of the 𝑖-th highest predicted label according to the output probability 𝛑 𝑑 .…”

Section: Experiments 41 Setupmentioning

confidence: 99%

MATCH: Metadata-Aware Text Classification in A Large Hierarchy

Shen

Dong

Wang

2021

Proceedings of the Web Conference 2021

View full text Add to dashboard Cite

Multi-label text classification refers to the problem of assigning each given document its most relevant labels from a label set. Commonly, the metadata of the given documents and the hierarchy of the labels are available in real-world applications. However, most existing studies focus on only modeling the text information, with a few attempts to utilize either metadata or hierarchy signals, but not both of them. In this paper, we bridge the gap by formalizing the problem of metadata-aware text classification in a large label hierarchy (e.g., with tens of thousands of labels). To address this problem, we present the MATCH 1 solution-an end-to-end framework that leverages both metadata and hierarchy information. To incorporate metadata, we pre-train the embeddings of text and metadata in the same space and also leverage the fully-connected attentions to capture the interrelations between them. To leverage the label hierarchy, we propose different ways to regularize the parameters and output probability of each child label by its parents. Extensive experiments on two massive text datasets with large-scale label hierarchies demonstrate the effectiveness of MATCH over the stateof-the-art deep learning baselines.

show abstract

“…propose a multi-label reasoner mechanism that employs multiple rounds of predictions, and relies on predicting multiple rounds of results to ensemble or determine a proper order, which is computationally expensive. CorNet-BertXML (Xun et al, 2020) utilizes BERT (Devlin et al, 2019) to obtain the joint representation of text and all candidate labels and extra exponential linear units (ELU) at the prediction layer to make use of label correlation knowledge. Different from the above works, we exploit extra label co-occurrence prediction tasks to explicitly model the label correlations in a multi-task framework.…”

Section: Label Correlation Learningmentioning

confidence: 99%

Enhancing Label Correlation Feedback in Multi-Label Text Classification via Multi-Task Learning

Zhang¹,

Zhang²,

Yan³

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

In multi-label text classification (MLTC), each given document is associated with a set of correlated labels. To capture label correlations, previous classifier-chain and sequenceto-sequence models transform MLTC to a sequence prediction task. However, they tend to suffer from label order dependency, label combination over-fitting and error propagation problems. To address these problems, we introduce a novel approach with multi-task learning to enhance label correlation feedback. We first utilize a joint embedding (JE) mechanism to obtain the text and label representation simultaneously. In MLTC task, a document-label cross attention (CA) mechanism is adopted to generate a more discriminative document representation. Furthermore, we propose two auxiliary label co-occurrence prediction tasks to enhance label correlation learning: 1) Pairwise Label Co-occurrence Prediction (PLCP), and 2) Conditional Label Co-occurrence Prediction (CLCP). Experimental results on AAPD and RCV1-V2 datasets show that our method outperforms competitive baselines by a large margin. We analyze low-frequency label performance, label dependency, label combination diversity and coverage speed to show the effectiveness of our proposed method on label correlation learning. Our code is available at https://github.com/EiraZhang/LACO. * Equal contribution. † Work done during an internship at Tencent.

show abstract

Correlation Networks for Extreme Multi-label Text Classification

Cited by 35 publications

References 16 publications

Beyond Text: Incorporating Metadata and Label Structure for Multi-Label Document Classification using Heterogeneous Graphs

Beyond Text: Incorporating Metadata and Label Structure for Multi-Label Document Classification using Heterogeneous Graphs

MATCH: Metadata-Aware Text Classification in A Large Hierarchy

Enhancing Label Correlation Feedback in Multi-Label Text Classification via Multi-Task Learning

Contact Info

Product

Resources

About