Enhancing Aspect-Based Sentiment Analysis With Capsule Network

Su, Jindian; Yu, Shujuan; Luo, Da

doi:10.1109/access.2020.2997675

Cited by 29 publications

(7 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Techniques such as layer normalization [261], residual connections [262], and attention mechanisms inherent in the transformer architecture further enhance BERT's Net [263] is a generalized autoregressive [264] pretraining method that surpasses the limitations of traditional left-to-right or right-to-left language modeling. XLNet is trained using a permutation-based approach that differs from traditional autoregressive models [265].…”

Section: B Training Of Llmsmentioning

confidence: 99%

Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects

Hadi,

tashi,

Qureshi

et al. 2023

Preprint

View full text Add to dashboard Cite

<p>Within the vast expanse of computerized language processing, a revolutionary entity known as Large Language Models (LLMs) has emerged, wielding immense power in its capacity to comprehend intricate linguistic patterns and conjure coherent and contextually fitting responses. Large language models (LLMs) are a type of artificial intelligence (AI) that have emerged as powerful tools for a wide range of tasks, including natural language processing (NLP), machine translation, and question-answering. This survey paper provides a comprehensive overview of LLMs, including their history, architecture, training methods, applications, and challenges. The paper begins by discussing the fundamental concepts of generative AI and the architecture of generative pre- trained transformers (GPT). It then provides an overview of the history of LLMs, their evolution over time, and the different training methods that have been used to train them. The paper then discusses the wide range of applications of LLMs, including medical, education, finance, and engineering. It also discusses how LLMs are shaping the future of AI and how they can be used to solve real-world problems. The paper then discusses the challenges associated with deploying LLMs in real-world scenarios, including ethical considerations, model biases, interpretability, and computational resource requirements. It also highlights techniques for enhancing the robustness and controllability of LLMs, and addressing bias, fairness, and generation quality issues. Finally, the paper concludes by highlighting the future of LLM research and the challenges that need to be addressed in order to make LLMs more reliable and useful. This survey paper is intended to provide researchers, practitioners, and enthusiasts with a comprehensive understanding of LLMs, their evolution, applications, and challenges. By consolidating the state-of-the-art knowledge in the field, this survey serves as a valuable resource for further advancements in the development and utilization of LLMs for a wide range of real-world applications. The GitHub repo for this project is available at https://github.com/anas-zafar/LLM-Survey</p>

show abstract

Section: B Training Of Llmsmentioning

confidence: 99%

Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects

Hadi,

tashi,

Qureshi

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The experimental results on six classification tasks indicate the effectiveness of the model and the algorithm helps to reduce the interference among tasks. Su et al [8] combined XLNet and capsule network to address the challenge of aspect-based sentiment analysis (ABSA); a capsule network with a dynamic routing algorithm was utilized to extract the local and spatial hierarchical relations of the text sequence and yield its local feature representations. Lin et al [9] first employed the conventional source-target attention to produce a timestep-specific sourceside context vector and fed the vector into a novel dynamic context-guided capsule network (DCCN) for multimodal machine translation (MMT), which achieved superior results on Multi30K dataset.…”

Section: Capsule Neural Network For Nlpmentioning

confidence: 99%

HAMCap: A Weak-Supervised Hybrid Attention-Based Capsule Neural Network for Fine-Grained Climate Change Debate Analysis

Xiang,

Fujii

2023

BDCC

View full text Add to dashboard Cite

Climate change (CC) has become a central global topic within the multiple branches of social disciplines. Natural Language Processing (NLP) plays a superior role since it has achieved marvelous accomplishments in various application scenarios. However, CC debates are ambiguous and complicated to interpret even for humans, especially when it comes to the aspect-oriented fine-grained level. Furthermore, the lack of large-scale effective labeled datasets is always a plight encountered in NLP. In this work, we propose a novel weak-supervised Hybrid Attention Masking Capsule Neural Network (HAMCap) for fine-grained CC debate analysis. Specifically, we use vectors with allocated different weights instead of scalars, and a hybrid attention mechanism is designed in order to better capture and represent information. By randomly masking with a Partial Context Mask (PCM) mechanism, we can better construct the internal relationship between the aspects and entities and easily obtain a large-scale generated dataset. Considering the uniqueness of linguistics, we propose a Reinforcement Learning-based Generator-Selector mechanism to automatically update and select data that are beneficial to model training. Empirical results indicate that our proposed ensemble model outperforms baselines on downstream tasks with a maximum of 50.08% on accuracy and 49.48% on F1 scores. Finally, we draw interpretable conclusions about the climate change debate, which is a widespread global concern.

show abstract

“…Simple attention networks can generate noisy features as well. This problem can be resolved using Capsule networks [31,32]. The capsule network can dynamically route the spatial features from the lower layer to the upper layers.…”

Section: Recent Trends In Aspect Level Sentiment Classificationmentioning

confidence: 99%

Benchmarking Deep Learning Methods for Aspect Level Sentiment Classification

Sharma

Kaur

2021

Applied Sciences

View full text Add to dashboard Cite

With the advancements in processing units and easy availability of cloud-based GPU servers, many deep learning-based methods have been proposed for Aspect Level Sentiment Classification (ALSC) literature. With this increase in the number of deep learning methods proposed in ALSC literature, it has become difficult to ascertain the performance difference of one method over the other. To this end, our study provides a statistical comparison of the performance of 35 recent deep learning methods with respect to three performance metrics-Accuracy, Macro F1 score, and Time. The methods are evaluated for eight benchmark datasets. In this study, the statistical comparison is based on Friedman, Nemenyi, and Wilcoxon tests. As per the results of statistical tests, the top-ranking methods could not significantly outperform several other methods in terms of Accuracy and Macro F1 score and performed poorly on-time metric. However, the time taken by any method is crucial to analyze the overall performance. Thus, this study aids the selection of the Deep Learning method, which maximizes the accuracy and Macro F1 score and takes minimal time. Our study also establishes a framework for validating the performance of new and alternate methods in ALSC that can be helpful for researchers and practitioners working in this area.

show abstract

Enhancing Aspect-Based Sentiment Analysis With Capsule Network

Cited by 29 publications

References 24 publications

Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects

Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects

HAMCap: A Weak-Supervised Hybrid Attention-Based Capsule Neural Network for Fine-Grained Climate Change Debate Analysis

Benchmarking Deep Learning Methods for Aspect Level Sentiment Classification

Contact Info

Product

Resources

About