Tree-to-Sequence Attentional Neural Machine Translation

Abstract

Most of the existing Neural Machine Translation (NMT) models focus on the conversion of sequential data and do not directly use syntactic information. We propose a novel end-to-end syntactic NMT model, extending a sequenceto-sequence model with the source-side phrase structure. Our model has an attention mechanism that enables the decoder to generate a translated word while softly aligning it with phrases as well as words of the source sentence. Experimental results on the WAT'15 Englishto-Japanese dataset demonstrate that our proposed model considerably outperforms sequence-to-sequence attentional NMT models and compares favorably with the state-of-the-art tree-to-string SMT system.

Sequence-to-Dependency Neural Machine Translation

“…Some effort has been done to incorporate source syntax into NMT. Eriguchi et al (2016) proposed a tree-to-sequence attentional NMT model where source-side parse tree was used and achieved promising improvement. Intuitively, adding source syntactic information to [Source] 只有 施工 人员 的 安全 得到 了 保证 , 才能 继续 施 工 .…”

A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks

“…For example, Part-Of-Speech (POS) tags are used for syntactic parsers. The parsers are used to improve higher-level tasks, such as natural language inference (Chen et al, 2016) and machine translation (Eriguchi et al, 2016). These systems are often pipelines and not trained end-to-end.…”

Chunk-based Decoder for Neural Machine Translation

“…The experimental results show that our best model outperforms the best single NMT model reported in WAT '16 (Eriguchi et al, 2016b).…”

“…Eriguchi et al (2016a)'s baseline system (the first line in Table 3) was the best single (w/o ensembling) word-based NMT system that were reported in WAT '16. For a more fair evaluation, we also reimplemented a standard attention-based NMT system that uses exactly the same encoder, training procedure, and the hyperparameters as our proposed models, but has a word-based decoder.…”

“…After the encoder maps a source sentence into a fixed-length vector, the decoder maps the vector into a target sentence. The implementation of the encoder can be a convolutional neural network (CNN) (Kalchbrenner and Blunsom, 2013), a long short-term memory (LSTM) (Sutskever et al, 2014;Luong and Manning, 2016), a gated recurrent unit (GRU) (Cho et al, 2014b;Bahdanau et al, 2015), or a Tree-LSTM (Eriguchi et al, 2016b). While various architectures are leveraged as an encoder to capture the structural information in the source language, most of the NMT models rely on a standard sequential network such as LSTM or GRU as the decoder.…”