Hyperbolic Capsule Networks for Multi-Label Classification

Chen, Boli; Huang, Xin; Xiao, Lin; Jing, Liping

doi:10.18653/v1/2020.acl-main.283

Cited by 19 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, Ganea et al [12] and Nickel & Kiela [30] introduce the basic operations of neural networks in the Poincaré ball and the Lorentz model respectively. After that, researchers further introduce various types of neural models in hyperbolic space including hyperbolic attention networks [14], hyperbolic graph neural networks [23,6], hyperbolic prototypical networks [26] and hyperbolic capsule networks [9]. Recently, with the rapid development of hyperbolic neural networks, people attempt to utilize them in various downstream tasks such as word embeddings [37], knowledge graph embeddings [7], entity typing [24], text classification [44], question answering [36] and machine translation [14,34], to handle their non-Euclidean properties, and have achieved significant and consistent improvement compared to the traditional neural models in Euclidean space.…”

Section: Related Workmentioning

confidence: 99%

Fully Hyperbolic Neural Networks

Chen¹,

Han²,

Lin³

et al. 2021

Preprint

View full text Add to dashboard Cite

Hyperbolic neural networks have shown great potential for modeling complex data. However, existing hyperbolic networks are not completely hyperbolic, as they encode features in a hyperbolic space yet formalize most of their operations in the tangent space (a Euclidean subspace) at the origin of the hyperbolic space. This hybrid method greatly limits the modeling ability of networks. In this paper, we propose a fully hyperbolic framework to build hyperbolic networks based on the Lorentz model by adapting the Lorentz transformations (including boost and rotation) to formalize essential operations of neural networks. Moreover, we also prove that linear transformation in tangent spaces used by existing hyperbolic networks is a relaxation of the Lorentz rotation and does not include the boost, implicitly limiting the capabilities of existing hyperbolic networks. The experimental results on four NLP tasks show that our method has better performance for building both shallow and deep networks. Our code will be released to facilitate follow-up research. * Equal contribution. 2 Both the Poincaré ball model and the Lorentz model are typical geometric models in hyperbolic geometry.Preprint. Under review.

show abstract

Section: Related Workmentioning

confidence: 99%

Fully Hyperbolic Neural Networks

Chen¹,

Han²,

Lin³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Capsule networks were proposed in [5] to learn the part-whole relationships of objects through iterative routing among different level capsules. The pipelines using capsule networks have achieved state-of-the-art results in some areas of Natural Language Processing (NLP) such as intent detection [13] and multi-label classification [14], as well as computer vision such as expression recognition [15] and low resolution image recognition [16]. Recently, an LSTM-CapsNet architecture was successfully proposed for EEG-based affective computing [6].…”

Section: Related Workmentioning

confidence: 99%

Distilling EEG Representations via Capsules for Affective Computing

Zhang¹,

Etemad²

2021

Preprint

View full text Add to dashboard Cite

Affective computing with Electroencephalogram (EEG) is a challenging task that requires cumbersome models to effectively learn the information contained in large-scale EEG signals, causing difficulties for real-time smart-device deployment. In this paper, we propose a novel knowledge distillation pipeline to distill EEG representations via capsule-based architectures for both classification and regression tasks. Our goal is to distill information from a heavy model to a lightweight model for subject-specific tasks. To this end, we first pre-train a large model (teacher network) on large number of training samples. Then, we employ the teacher network to learn the discriminative features embedded in capsules by adopting a lightweight model (student network) to mimic the teacher using the privileged knowledge. Such privileged information learned by the teacher contain similarities among capsules and are only available during the training stage of the student network. We evaluate the proposed architecture on two large-scale public EEG datasets, showing that our framework consistently enables student networks with different compression ratios to effectively learn from the teacher, even when provided with limited training samples. Lastly, our method achieves state-of-the-art results on one of the two datasets.

show abstract

“…These approaches may not be easily extended to those tasks without external knowledge. To this end, the multi-label text classification approaches can be quickly applied to emotion recognition (Chen et al 2020;Fei et al 2020). Recently, Yang et al (2019) leverage a reinforced approach to find a better sequence than a baseline sequence, but it still relies on the pre-trained seq2seq model with a pre-defined order of ground-truth.…”

Section: Related Workmentioning

confidence: 99%

Multi-modal Multi-label Emotion Recognition with Heterogeneous Hierarchical Message Passing

Zhang¹,

Ju²,

Zhang

et al. 2021

AAAI

View full text Add to dashboard Cite

As an important research issue in affective computing community, multi-modal emotion recognition has become a hot topic in the last few years. However, almost all existing studies perform multiple binary classification for each emotion with focus on complete time series data. In this paper, we focus on multi-modal emotion recognition in a multi-label scenario. In this scenario, we consider not only the label-to-label dependency, but also the feature-to-label and modality-to-label dependencies. Particularly, we propose a heterogeneous hierarchical message passing network to effectively model above dependencies. Furthermore, we propose a new multi-modal multi-label emotion dataset based on partial time-series content to show predominant generalization of our model. Detailed evaluation demonstrates the effectiveness of our approach.

show abstract

Hyperbolic Capsule Networks for Multi-Label Classification

Cited by 19 publications

References 15 publications

Fully Hyperbolic Neural Networks

Fully Hyperbolic Neural Networks

Distilling EEG Representations via Capsules for Affective Computing

Multi-modal Multi-label Emotion Recognition with Heterogeneous Hierarchical Message Passing

Contact Info

Product

Resources

About