Fine-grained attention mechanism for neural machine translation

Choi, Heeyoul; Cho, Kyunghyun; Bengio, Yoshua

doi:10.1016/j.neucom.2018.01.007

Cited by 167 publications

(60 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where l is the index number of operation results and ranges from 1 to L. Then, the neural attention mechanism 43 is utilized as a part of decoder to generate prediction results. We rstly compute two weight factors in neural attention mechanism as the following formulas:…”

Section: Decodingmentioning

confidence: 99%

Data-driven prediction and control of wastewater treatment process through the combination of convolutional neural network and recurrent neural network

Guo

Wang

et al. 2020

RSC Adv.

View full text Add to dashboard Cite

show abstract

Section: Decodingmentioning

confidence: 99%

Data-driven prediction and control of wastewater treatment process through the combination of convolutional neural network and recurrent neural network

Guo

Wang

et al. 2020

RSC Adv.

View full text Add to dashboard Cite

show abstract

“…(13), the attention score a i jk =1.0, leading to e i j = square(r i jk ), which is similar to Eq. (5). In this case, SDM will measure the distance between target item j and context item k in the same way as SDP model does.…”

Section: Attention Modulementioning

confidence: 99%

Signed Distance-based Deep Memory Recommender

Tran

Lee

Kong

2019

The World Wide Web Conference

View full text Add to dashboard Cite

Personalized recommendation algorithms learn a user's preference for an item by measuring a distance/similarity between them. However, some of the existing recommendation models (e.g., matrix factorization) assume a linear relationship between the user and item. This approach limits the capacity of recommender systems, since the interactions between users and items in real-world applications are much more complex than the linear relationship. To overcome this limitation, in this paper, we design and propose a deep learning framework called Signed Distance-based Deep Memory Recommender, which captures non-linear relationships between users and items explicitly and implicitly, and work well in both general recommendation task and shopping basket-based recommendation task. Through an extensive empirical study on six real-world datasets in the two recommendation tasks, our proposed approach achieved significant improvement over ten state-of-the-art recommendation models. KEYWORDSMemory recommender; signed distance; metric-based attention.

show abstract

“…The autoencoder (AE) (Vincent et al 2008) is one of the most popular unsupervised neural network approaches. It has been widely used as a performant mechanism to pre-train neural networks and general purpose feature learning (Choi et al 2018). It allows to compress the representation of input data, disentangling the main factors of variability, removing redundancies and reducing the dimension of the input.…”

Section: Autoencodermentioning

confidence: 99%

Unsupervised network embeddings with node identity awareness

Gutiérrez-Gómez

Delvenne

2019

Appl Netw Sci

View full text Add to dashboard Cite

A main challenge in mining network-based data is finding effective ways to represent or encode graph structures so that it can be efficiently exploited by machine learning algorithms. Several methods have focused in network representation at node/edge or substructure level. However, many real life challenges related with time-varying, multilayer, chemical compounds and brain networks involve analysis of a family of graphs instead of single one opening additional challenges in graph comparison and representation. Traditional approaches for learning representations relies on hand-crafted specialized features to extract meaningful information about the graphs, e.g. statistical properties, structural motifs, etc. as well as popular graph distances to quantify dissimilarity between networks. In this work we provide an unsupervised approach to learn graph embeddings for a collection of graphs defined on the same set of nodes so that it can be used in numerous graph mining tasks. By using an unsupervised neural network approach on input graphs, we aim to capture the underlying distribution of the data in order to discriminate between different class of networks. Our method is assessed empirically on synthetic and real life datasets and evaluated in three different tasks: graph clustering, visualization and classification. Results reveal that our method outperforms well known graph distances and graph-kernels in clustering and classification tasks, being highly efficient in runtime.

show abstract

Fine-grained attention mechanism for neural machine translation

Cited by 167 publications

References 16 publications

Data-driven prediction and control of wastewater treatment process through the combination of convolutional neural network and recurrent neural network

Data-driven prediction and control of wastewater treatment process through the combination of convolutional neural network and recurrent neural network

Signed Distance-based Deep Memory Recommender

Unsupervised network embeddings with node identity awareness

Contact Info

Product

Resources

About