Multi-layer Attention Based CNN for Target-Dependent Sentiment Classification

Zhang, Suqi; Xu, Xinyun; Pang, Yanwei; Han, Jungong

doi:10.1007/s11063-019-10017-9

Cited by 36 publications

(21 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yang et al [18] applied an attention mechanism to bidirectional LSTM in two models for target-dependent sentiment classification of a Twitter dataset and achieved improvement over baseline techniques. Zhang et al [19] also approached target-dependent sentiment classification using a multi-layer CNN with an attention mechanism that modeled context representation and achieved high accuracy on product reviews and Twitter data. Gan et al [20] designed a dilated CNN based on sparse attention to perform targeted sentiment analysis, which identified sentiment orientation and handled complex sequences.…”

Section: Literature Reviewmentioning

confidence: 99%

An attention-based CNN-LSTM model for subjectivity detection in opinion-mining

Sagnika

Mishra

Meher

2021

Neural Comput & Applic

View full text Add to dashboard Cite

Opinion-mining generally refers to analyzing opinions on various topics available in the form of text. It is an essential operation of natural language processing since it enables efficient decision-making and planning for users and businesses. Opinion-mining can be made more comfortable and more effective by initially performing subjectivity detection, i.e., identifying the text as subjective or objective. An opinion-mining model can better identify the opinions present in the remaining subjective statements by removing objective statements. With this reasoning, we present an efficient subjectivity detection model for improved accuracy in Opinion-mining. The model uses a strategic combination of convolutional neural network (CNN) and long short-term memory (LSTM). CNN and LSTM are state-of-the-art deep learning models that can efficiently process textual data and identify inherent connections and patterns with varying abstraction levels. The proposed work combines the strengths of both these models in an ensemble model. Effectiveness of the model is enhanced with the incorporation of an attention network. In the present task, the sentences are represented as word embeddings that include sentiment information and part-of-speech. The proposed model is applied on two movie review datasets, and its performance is evaluated compared with state-of-the-art methods. Various performance indexes have validated the superiority of the proposed model in the opinion-mining task.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

An attention-based CNN-LSTM model for subjectivity detection in opinion-mining

Sagnika

Mishra

Meher

2021

Neural Comput & Applic

View full text Add to dashboard Cite

show abstract

“…Although RNN is suitable for processing time-series information, it has the problem of gradient explosion or gradient disappearance [34], [35]. To solve this problem, LSTM introduces a gating mechanism into the traditional network structure of RNN to support the long-term dependence of sequence information [36], [37]. In natural language tasks, since the text itself has the property of sequence, it is very suitable to use LSTM for processing theoretically.…”

Section: Related Workmentioning

confidence: 99%

Multi-Task Learning Model Based on Multi-Scale CNN and LSTM for Sentiment Classification

Jin

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Sentiment classification is an interesting and crucial research topic in the field of natural language processing (NLP). Data-driven methods, including machine learning and deep learning techniques, provide one direct and effective solution to solve the sentiment classification problem. However, the classification performance declines when the input includes review comments for multiple tasks. The most appropriate way of constructing a sentiment classification model under multi-tasking circumstances remains questionable in the related field. In this study, aiming at the multi-tasking sentiment classification problem, we propose a multi-task learning model based on a multi-scale convolutional neural network (CNN) and long short term memory (LSTM) for multi-task multi-scale sentiment classification (MTL-MSCNN-LSTM). The model comprehensively utilizes and properly handles global features and local features of different scales of text to model and represent sentences. The multi-task learning framework improves the encoder quality, simultaneously improving the results of emotion classification. Six different types of commodity review datasets were employed in the experiment. Using accuracy and F1-score as the metrics to evaluate the performance of the proposed model, comparing with methods such as single-task learning and LSTM encoder, the proposed MTL-MSCNN-LSTM model outperforms most of the existing methods.INDEX TERMS Sentiment classification, multi-task learning model, long short term memory, multi-scale convolutional neural network.

show abstract

“…Meanwhile, attention mechanism is an effective method, and its development renders the outputs of models more interpretable. Recently, they have often been combined with deep learning methods and successfully applied to multiple fields [22,33,35,40].…”

Section: Introductionmentioning

confidence: 99%

Multi-layer Representation Learning and Its Application to Electronic Health Records

Yang

Zheng

et al. 2021

Neural Process Lett

View full text Add to dashboard Cite

Electronic Health Records (EHRs) are digital records associated with hospitalization, diagnosis, medications and so on. Secondary use of EHRs can promote the clinical informatics applications and the development of healthcare undertaking. EHRs have the unique characteristic where the patient visits are temporally ordered but the diagnosis codes within a visit are randomly ordered. The hierarchical structure requires a multi-layer network to explore the different relational information of EHRs. In this paper, we propose a Multi-Layer Representation Learning method (MLRL), which is capable of learning effective patient representation by hierarchically exploring the valuable information in both diagnosis codes and patient visits. Firstly, MLRL utilizes the multi-head attention mechanism to explore the potential connections in diagnosis codes, and a linear transformation is implemented to further map the code vectors to non-negative real-valued representations. The initial visit vectors are then obtained by summarizing all the code representations. Secondly, the proposed method combines Bidirectional Long Short-Term Memory with self-attention mechanism to learn the weighted visit vectors which are aggregated to form the patient representation. Finally, to evaluate the performance of MLRL, we apply it to patient's mortality prediction on real EHRs and the experimental results demonstrate that MLRL has a significant improvement in prediction performance. MLRL achieves around 0.915 in Area Under Curve which is superior to the results obtained by baseline methods. Furthermore, compared with raw data and other data representations, the learned representation with MLRL shows its outstanding results and availability on multiple different classifiers.

show abstract

Multi-layer Attention Based CNN for Target-Dependent Sentiment Classification

Cited by 36 publications

References 34 publications

An attention-based CNN-LSTM model for subjectivity detection in opinion-mining

An attention-based CNN-LSTM model for subjectivity detection in opinion-mining

Multi-Task Learning Model Based on Multi-Scale CNN and LSTM for Sentiment Classification

Multi-layer Representation Learning and Its Application to Electronic Health Records

Contact Info

Product

Resources

About