Attention augmentation with multi-residual in bidirectional LSTM

Wang, Ye; Zhang, Xinxiang; Lu, Mi; Wang, Han; Choe, Yoonsuck

doi:10.1016/j.neucom.2019.10.068

Cited by 36 publications

(15 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ScSE calculated spatial attention using 2D convolution and then combined it with channel attention [31]. Wang Y. et al proposed methods of time-series data (including text and video) classification using LSTM with multi-residual attention mechanism [32,33]. In A2-Net, a new method for image or video recognition based on NL block relation function was introduced [34].…”

Section: Related Work 21 Attention Mechanismmentioning

confidence: 99%

A Cascade Attention Based Facial Expression Recognition Network by Fusing Multi-Scale Spatio-Temporal Features

Zhu

Zhao

et al. 2022

Sensors

View full text Add to dashboard Cite

The performance of a facial expression recognition network degrades obviously under situations of uneven illumination or partial occluded face as it is quite difficult to pinpoint the attention hotspots on the dynamically changing regions (e.g., eyes, nose, and mouth) as precisely as possible. To address the above issue, by a hybrid of the attention mechanism and pyramid feature, this paper proposes a cascade attention-based facial expression recognition network on the basis of a combination of (i) local spatial feature, (ii) multi-scale-stereoscopic spatial context feature (extracted from the 3-scale pyramid feature), and (iii) temporal feature. Experiments on the CK+, Oulu-CASIA, and RAF-DB datasets obtained recognition accuracy rates of 99.23%, 89.29%, and 86.80%, respectively. It demonstrates that the proposed method outperforms the state-of-the-art methods in both the experimental and natural environment.

show abstract

Section: Related Work 21 Attention Mechanismmentioning

confidence: 99%

A Cascade Attention Based Facial Expression Recognition Network by Fusing Multi-Scale Spatio-Temporal Features

Zhu

Zhao

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…The residual connection is used between the two Bi-LSTM layers. It allows the gradients to pass through the network directly and also helps to preserve the long range dependencies [14], [15]. We have tested our proposed approach with a test data set and the performance is compared with other DNER models and found to improve the recognition rate over the preceding state-of-the-art models.…”

Section: Introductionmentioning

confidence: 99%

A novel deep learning architecture for drug named entity recognition

Mathu

Raimond

2021

TELKOMNIKA

View full text Add to dashboard Cite

Drug named entity recognition (DNER) becomes the prerequisite of other medical relation extraction systems. Existing approaches to automatically recognize drug names includes rule-based, machine learning (ML) and deep learning (DL) techniques. DL techniques have been verified to be the stateof-the-art as it is independent of handcrafted features. The previous DL methods based on word embedding input representation uses the same vector representation for an entity irrespective of its context in different sentences and hence could not capture the context properly. Also, identification of the n-gram entity is a challenge. In this paper, a novel architecture is proposed that includes a sentence embedding layer that works on the entire sentence to efficiently capture the context of an entity. A hybrid model that comprises a stacked bidirectional long short-term memory (Bi-LSTM) with residual LSTM has been designed to overcome the limitations and to upgrade the performance of the model. We have contrasted the achievement of our proposed approach with other DNER models and the percentage of improvements of the proposed model over LSTM-conditional random field (CRF), LIU and WBI with respect to micro-average F1-score are 11.17, 8.8 and 17.64 respectively. The proposed model has also shown promising result in recognizing 2-and 3-gram entities.

show abstract

“…Deep learning technology can automatically learn high-level representations from raw data, which overcomes the shortcomings mentioned above. Therefore, it has been applied widely in many fields such as image processing, natural language processing, and bioinformatics [20][21][22][23]. For predicting succinylation sites, Huang et al used position-specific amino-acid composition, the composition of k-spaced amino-acid pairs, and a positionspecific scoring matrix to characterize original sequences [24].…”

Section: Introductionmentioning

confidence: 99%

MDCAN-Lys: A Model for Predicting Succinylation Sites Based on Multilane Dense Convolutional Attention Network

et al. 2021

View full text Add to dashboard Cite

Lysine succinylation is an important post-translational modification, whose abnormalities are closely related to the occurrence and development of many diseases. Therefore, exploring effective methods to identify succinylation sites is helpful for disease treatment and research of related drugs. However, most existing computational methods for the prediction of succinylation sites are still based on machine learning. With the increasing volume of data and complexity of feature representations, it is necessary to explore effective deep learning methods to recognize succinylation sites. In this paper, we propose a multilane dense convolutional attention network, MDCAN-Lys. MDCAN-Lys extracts sequence information, physicochemical properties of amino acids, and structural properties of proteins using a three-way network, and it constructs feature space. For each sub-network, MDCAN-Lys uses the cascading model of dense convolutional block and convolutional block attention module to capture feature information at different levels and improve the abstraction ability of the network. The experimental results of 10-fold cross-validation and independent testing show that MDCAN-Lys can recognize more succinylation sites, which is consistent with the conclusion of the case study. Thus, it is worthwhile to explore deep learning-based methods for the recognition of succinylation sites.

show abstract

Attention augmentation with multi-residual in bidirectional LSTM

Cited by 36 publications

References 8 publications

A Cascade Attention Based Facial Expression Recognition Network by Fusing Multi-Scale Spatio-Temporal Features

A Cascade Attention Based Facial Expression Recognition Network by Fusing Multi-Scale Spatio-Temporal Features

A novel deep learning architecture for drug named entity recognition

MDCAN-Lys: A Model for Predicting Succinylation Sites Based on Multilane Dense Convolutional Attention Network

Contact Info

Product

Resources

About