Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications

Zhao, Wei; Peng, Haiyun; Eger, Steffen; Wang, Zhaoxia; Yang, Min

doi:10.18653/v1/p19-1150

Cited by 118 publications

(51 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For example, Tang et al [58] used a convolutional neural network (CNN) to obtain word embeddings for words frequently used in tweets and dos Santos and Gatti [17] employed a deep CNN for sentiment detection in short texts. More recent approaches have been focusing on the development of sentiment-specific word embeddings [44], which are able to encode more affective clues than regular word vectors, and on the use of context-aware subsymbolic approaches such as attention modeling [32,33] and capsule networks [13,66].…”

Section: Related Workmentioning

confidence: 99%

SenticNet 6: Ensemble Application of Symbolic and Subsymbolic AI for Sentiment Analysis

Wang

Xing

et al. 2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

Self Cite

369

144

View full text Add to dashboard Cite

Deep learning has unlocked new paths towards the emulation of the peculiarly-human capability of learning from examples. While this kind of bottom-up learning works well for tasks such as image classification or object detection, it is not as effective when it comes to natural language processing. Communication is much more than learning a sequence of letters and words: it requires a basic understanding of the world and social norms, cultural awareness, commonsense knowledge, etc.; all things that we mostly learn in a top-down manner. In this work, we integrate top-down and bottom-up learning via an ensemble of symbolic and subsymbolic AI tools, which we apply to the interesting problem of polarity detection from text. In particular, we integrate logical reasoning within deep learning architectures to build a new version of Sentic-Net, a commonsense knowledge base for sentiment analysis.

show abstract

Section: Related Workmentioning

confidence: 99%

SenticNet 6: Ensemble Application of Symbolic and Subsymbolic AI for Sentiment Analysis

Wang

Xing

et al. 2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

Self Cite

369

144

View full text Add to dashboard Cite

show abstract

“…Recently, deep learning-based aspect extraction [52], attention-based LSTM [53] and capsule networks [54,55] have also been widely used for sentiment analysis and challenging NLP applications that yield state-of-the-art prediction results. In this work, our focus is on the design of a unified sentiment analysis framework for multi-domain textual sentiment classification.…”

Section: Related Workmentioning

confidence: 99%

LeSSA: A Unified Framework based on Lexicons and Semi-Supervised Learning Approaches for Textual Sentiment Classification

Khan

Lee

2019

Applied Sciences

View full text Add to dashboard Cite

Sentiment Analysis (SA) is an active research area. SA aims to classify the online unstructured user-generated contents (UUGC) into positive and negative classes. A reliable training data is vital to learn a sentiment classifier for textual sentiment classification, but due to domain heterogeneity, manually construction of reliable labeled sentiment corpora is a laborious and time-consuming task. In the absence of enough labeled data, the alternative usage of sentiment lexicons and semi-supervised learning approaches for sentiment classification have substantially attracted the attention of the research community. However, state-of-the-art techniques for semi-supervised sentiment classification present research challenges expressed in questions like the following. How to effectively utilize the concealed significant information in the unstructured data? How to learn the model while considering the most effective sentiment features? How to remove the noise and redundant features? How to refine the initial training data for initial model learning as the random selection may lead to performance degradation? Besides, mainly existing lexicons have trouble with word coverage, which may ignore key domain-specific sentiment words. Further research is required to improve the sentiment lexicons for textual sentiment classification. In order to address such research issues, in this paper, we propose a novel unified sentiment analysis framework for textual sentiment classification called LeSSA. Our main contributions are threefold. (a) lexicon construction, generating quality and wide coverage sentiment lexicon. (b) training classification models based on a high-quality training dataset generated by using k-mean clustering, active learning, self-learning, and co-training algorithms. (c) classification fusion, whereby the predictions from numerous learners are confluences to determine final sentiment polarity based on majority voting, and (d) practicality, that is, we validate our claim while applying our model on benchmark datasets. The empirical evaluation of multiple domain benchmark datasets demonstrates that the proposed framework outperforms existing semi-supervised learning techniques in terms of classification accuracy.

show abstract

“…The output of capsule networks can serve as conventional probability of hypothesis while pose vector can bring specific interpretable information as auxiliary evidence to final decision. Also, since the extrapolation during affine transformation and generative routing method, capsule network can be learned with many fewer samples and prevail deep models [9], [10].…”

Section: Capsule Networkmentioning

confidence: 99%

“…4. Here we compared three methods: XML-CNN [14], NLPCap [9] and our proposed NLP-Cap with perturbed samples (NLP-Cap-Ge). The performances at two standard rank-based tasks, Precision@k and Normalized Discounted Cumulative Gain (NDCG@k), are considered.…”

Section: A Text Classification With Fewer Samplesmentioning

confidence: 99%

“…Differing from typical deep learning based models, capsule network integrates unsupervised routing with standard back propagation (BP) learning of neural networks. It helps model to extrapolate latent samples with the same distribution by affine transformation, and thus significantly prompt the generalization capacity especially on small datasets [9], [10].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Capsule Networks, Natural Language Processing, Deep Learning, Smart Grid, Reliability Evaluation

Wu¹,

Lu²,

Su³

et al. 2019

2019 Scientific Conference on Network, Power Systems and Computing (NPSC 2019)

View full text Add to dashboard Cite

To evaluate reliability of equipments plays a key role for the robustness of modern power communication network. Such tasks imply to learn a regression model with limited samples and heterogeneous formats. In this paper, we address this problem by introducing a deep learning based model containing the latest proposed capsule structures. The model can take raw maintenance data as input directly by treating them as unified document without any extra manual preprocessing. We propose a multiple stages strategy to train the model with original data as well as generated perturbed data from a decoder as augmentation. Experimental results demonstrate that not only the proposed method shows acceptable performance to predict the reliability of communication equipments, but also it shows potentiality especially in learning deep models with fewer samples in different Natural Language Processing tasks. Thus, the proposed capsule networks framework with data generation mechanism could be considered as a promising way to drive deep models in practical learning tasks in which only limited training data is available.

show abstract

Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications

Cited by 118 publications

References 40 publications

SenticNet 6: Ensemble Application of Symbolic and Subsymbolic AI for Sentiment Analysis

SenticNet 6: Ensemble Application of Symbolic and Subsymbolic AI for Sentiment Analysis

LeSSA: A Unified Framework based on Lexicons and Semi-Supervised Learning Approaches for Textual Sentiment Classification

Capsule Networks, Natural Language Processing, Deep Learning, Smart Grid, Reliability Evaluation

Contact Info

Product

Resources

About