Explicit Interaction Model towards Text Classification

Abstract

Text classification is one of the fundamental tasks in natural language processing. Recently, deep neural networks have achieved promising performance in the text classification task compared to shallow models. Despite of the significance of deep models, they ignore the fine-grained (matching signals between words and classes) classification clues since their classifications mainly rely on the text-level representations. To address this problem, we introduce the interaction mechanism to incorporate word-level matching signals into the text classification task. In particular, we design a novel framework, EXplicit interAction Model (dubbed as EXAM), equipped with the interaction mechanism. We justified the proposed approach on several benchmark datasets including both multilabel and multi-class text classification tasks. Extensive experimental results demonstrate the superiority of the proposed method. As a byproduct, we have released the codes and parameter settings to facilitate other researches.

Hierarchy-Aware Global Model for Hierarchical Text Classification

“…Hierarchy-Aware Multi-Label Attention The first variant of HiAGM is proposed based on multilabel attention, called as HiAGM-LA. Attention mechanism is usually utilized as the memory unit in text classification (Yang et al, 2016;Du et al, 2019). Recent LCL studies (Huang et al, 2019;You et al, 2019) construct one multi-label attentionbased model per level so as to avoid optimizing label embedding among different levels.…”

Hyperbolic Interaction Model for Hierarchical Multi-Label Classification

“…Baselines To demonstrate the effectiveness of HyperIM on the benchmark datasets, five comparative multi-label classification methods are chosen. EXAM (Du et al 2019) is the state-of-the-art interaction model for text classification. EXAM use pre-trained word embeddings in the Euclidean space, its label embeddings are randomly initialized.…”

“…; score(x T , l i )] can be calculate via certain score function, p [i] is then deduced from s i . This process is adapted from the interaction mechanism (Du et al 2019), which is usually used in tasks like natural language inference (Wang and Jiang 2016). Based on the idea that labels can be considered as abstraction from their word descriptions, sometimes a label is even a word itself, the word-label similarities can be derived from their embeddings in the latent space by the same way as the word similarity, which is widely studied in word embedding methods such as GloVe (Pennington, Socher, and Manning 2014).…”

“…The hyperbolic space can be described via Riemannian geometry (Hopper and Andrews 2011). Following previous works (Nickel and Kiela 2017;Ganea, Becigneul, and Hofmann 2018;Tifrea, Bécigneul, and Ganea 2019), we adopt the Poincaré ball.…”

“…With the purpose of generalizing operations for neural networks in the Poincaré ball, the formalism of the Möbius gyrovector space is used (Ganea, Becigneul, and Hofmann 2018). The Möbius addition for u, v ∈ B n is defined as…”