Tree-to-Sequence Attentional Neural Machine Translation

Eriguchi, Akiko; Hashimoto, Kazuma; Tsuruoka, Yoshimasa

doi:10.48550/arxiv.1603.06075

Cited by 28 publications

(14 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, many research efforts and the key contributions have been made to address the limitations of Seq2Seq when dealing with more complex data, that leverage external information using specialized neural models attached to underlying targeted applications, including Tree2Seq (Eriguchi et al, 2016), Set2Seq (Vinyals et al, 2015a), Recursive Neural Networks (Socher et al, 2010), and Tree-Structured LSTM (Tai et al, 2015). Due to more recent advances in graph representations and graph convolutional networks, a number of research has investigated to utilize various GNN to improve the performance over the Seq2Seq models in the domains of machine translation and graph generation (Bastings et al, 2017;Beck et al, 2018;Simonovsky & Komodakis, 2018;Li et al, 2018).…”

Section: Related Workmentioning

confidence: 99%

“…However, the Seq2Seq model often fails to perform as well as hoped on these problems, in part because it inevitably suffers significant information loss due to the conversion of complex structured data into a sequence, especially when the input data is naturally represented as graphs. Recently, a line of research efforts have been devoted to incorporate additional information by extracting syntactic information such as the phrase structure of a source sentence (Tree2seq) (Eriguchi et al, 2016), by utilizing attention mechanisms for input sets (Set2seq) (Vinyals et al, 2015a), and by encoding sentences recursively as trees (Socher et al, 2010;Tai et al, 2015). Although these methods achieve promising results on certain classes of problems, most of the presented techniques largely depend on the underlying application and may not be able to generalize to a broad class of problems in a general way.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks

Xu,

Wu,

Wang

et al. 2018

Preprint

View full text Add to dashboard Cite

The celebrated Sequence to Sequence learning (Seq2Seq) technique and its numerous variants achieve excellent performance on many tasks. However, many machine learning tasks have inputs naturally represented as graphs; existing Seq2Seq models face a significant challenge in achieving accurate conversion from graph form to the appropriate sequence. To address this challenge, we introduce a novel general end-to-end graph-to-sequence neural encoder-decoder model that maps an input graph to a sequence of vectors and uses an attention-based LSTM method to decode the target sequence from these vectors. Our method first generates the node and graph embeddings using an improved graph-based neural network with a novel aggregation strategy to incorporate edge direction information in the node embeddings. We further introduce an attention mechanism that aligns node embeddings and the decoding sequence to better cope with large graphs. Experimental results on bAbI, Shortest Path, and Natural Language Generation tasks demonstrate that our model achieves state-of-the-art performance and significantly outperforms existing graph neural networks, Seq2Seq, and Tree2Seq models; using the proposed bi-directional node embedding aggregation strategy, the model can converge rapidly to the optimal performance.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks

Xu,

Wu,

Wang

et al. 2018

Preprint

View full text Add to dashboard Cite

show abstract

“…The neural encoder-decoder models [17], [22] have been widely extended to model the mapping of general object inputs to their corresponding sequences [23], [24]. Recent advances in graph deep learning and graph convolutional networks have enabled various graph deep learning models to handle challenges in the domains of graph generation [25]- [27] and graph-to-sequence learning [28].…”

Section: Neural Encoder-decoder Modelsmentioning

confidence: 99%

DynGraph2Seq: Dynamic-Graph-to-Sequence Interpretable Learning for Health Stage Prediction in Online Health Forums

Gao

Homayoun

et al. 2019

2019 IEEE International Conference on Data Mining (ICDM)

View full text Add to dashboard Cite

Online health communities such as the online breast cancer forum enable patients (i.e., users) to interact and help each other within various subforums, which are subsections of the main forum devoted to specific health topics. The changing nature of the users' activities in different subforums can be strong indicators of their health status changes. This additional information could allow health-care organizations to respond promptly and provide additional help for the patient. However, modeling complex transitions of an individual user's activities among different subforums over time and learning how these correspond to his/her health stage are extremely challenging. In this paper, we first formulate the transition of user activities as a dynamic graph with multi-attributed nodes, then formalize the health stage inference task as a dynamic graph-to-sequence learning problem, and hence propose a novel dynamic graphto-sequence neural networks architecture (DynGraph2Seq) to address all the challenges. Our proposed DynGraph2Seq model consists of a novel dynamic graph encoder and an interpretable sequence decoder that learn the mapping between a sequence of time-evolving user activity graphs and a sequence of target health stages. We go on to propose dynamic graph hierarchical attention mechanisms to facilitate the necessary multi-level interpretability. A comprehensive experimental analysis of its use for a health stage prediction task demonstrates both the effectiveness and the interpretability of the proposed models.

show abstract

“…In [37], a tree-structured recursive neural network is applied for visual-textual sentiment analysis. The tree structures have also been utilized for machine translation in [8] and for object localization in [14]. Our model differs from them in the following aspects: We introduce a tree-structured memory network which contains the heterogeneous nodes.…”

Section: Tree Networkmentioning

confidence: 99%

A Better Way to Attend: Attention with Trees for Video Question Answering

Xue,

Chu,

Zhao

et al. 2019

Preprint

View full text Add to dashboard Cite

Tree-to-Sequence Attentional Neural Machine Translation

Cited by 28 publications

References 13 publications

Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks

Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks

DynGraph2Seq: Dynamic-Graph-to-Sequence Interpretable Learning for Health Stage Prediction in Online Health Forums

A Better Way to Attend: Attention with Trees for Video Question Answering

Contact Info

Product

Resources

About