An Analysis of Capsule Networks for Part of Speech Tagging in High- and Low-resource Scenarios

Abstract

Neural networks are a common tool in NLP, but it is not always clear which architecture to use for a given task. Different tasks, different languages, and different training conditions can all affect how a neural network will perform. Capsule Networks (CapsNets) are a relatively new architecture in NLP. Due to their novelty, CapsNets are being used more and more in NLP tasks. However, their usefulness is still mostly untested. In this paper, we compare three neural network architectures-LSTM, CNN, and CapsNet-on a part of speech tagging task. We compare these architectures in both high-and low-resource training conditions and find that no architecture consistently performs the best. Our analysis shows that our CapsNet performs nearly as well as a more complex LSTM under certain training conditions, but not others, and that our CapsNet almost always outperforms our CNN. We also find that our CapsNet implementation shows faster prediction times than the LSTM for Scottish Gaelic but not for Spanish, highlighting the effect that the choice of languages can have on the models.

Leveraging Capsule Routing to Associate Knowledge with Medical Literature Hierarchically

“…Relatively, the capsule network is newly proposed neural architecture in recent years and still being explored for its applications in NLP area (Zupon et al, 2020;Nguyen et al, 2019;Zhao et al, 2019). Several researches have explored to apply the capsule network to various NLP tasks, e,g., sentiment classification (Ke et al, 2021;Du et al, 2019b;Chen and Qian, 2019), relation extraction (Liu et al, 2020a), text classification (Chen et al, 2020;Du et al, 2019a;Xiao et al, 2018;Zhao et al, 2018), intent detection Xia et al, 2018), document translation , word sense disam-biguation (Liu et al, 2020b), etc.…”