Exploiting Semantics in Neural Machine Translation with Graph
            Convolutional Networks

Marcheggiani, Diego; Bastings, Jasmijn; Titov, Ivan

doi:10.18653/v1/n18-2078

Cited by 190 publications

(95 citation statements)

References 19 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using Full as the training data, the scores become 23.3, 23.9, 24.5 and 24.9, respectively. In addition to the different semantic representation being used (AMR vs SRL), Marcheggiani et al (2018) laid graph convolutional network (GCN) (Kipf and Welling, 2017) layers on top of a bidirectional LSTM (BiLSTM) layer, and then concatenated layer outputs as the attention memory. GCN layers encode the semantic role information, while BiLSTM layers encode the input sentence in the source language, and the concatenated hidden states of both layers contain information from both semantic role and source sentence.…”

Section: Resultsmentioning

confidence: 99%

“…On the other hand, exploring semantics for NMT has so far received relatively little attention. Recently, Marcheggiani et al (2018) exploited semantic role labeling (SRL) for NMT, showing that the predicate-argument information from SRL can improve the performance of an attentionbased sequence-to-sequence model by alleviating the "argument switching" problem, 1 one frequent 1 flipping arguments corresponding to different roles and severe issue faced by NMT systems (Isabelle et al, 2017). Figure 1 (a) shows one example of semantic role information, which only captures the relations between a predicate (gave) and its arguments (John, wife and present).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Semantic Neural Machine Translation Using AMR

Song

Gildea

Zhang

et al. 2019

Transactions of the Association for Computational Linguistics

102

View full text Add to dashboard Cite

It is intuitive that semantic representations can be useful for machine translation, mainly because they can help in enforcing meaning preservation and handling data sparsity (many sentences correspond to one meaning) of machine translation models. On the other hand, little work has been done on leveraging semantics for neural machine translation (NMT). In this work, we study the usefulness of AMR (short for abstract meaning representation) on NMT. Experiments on a standard English-to-German dataset show that incorporating AMR as additional knowledge can significantly improve a strong attention-based sequence-tosequence neural translation model.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Semantic Neural Machine Translation Using AMR

Song

Gildea

Zhang

et al. 2019

Transactions of the Association for Computational Linguistics

102

View full text Add to dashboard Cite

show abstract

“…Bastings et al [153] apply the Syntactic GCN to the task of neural machine translation. Marcheggiani et al [154] further adopt the same model as Bastings et al [153] to handle the semantic dependency graph of a sentence.…”

Section: Practical Applicationsmentioning

confidence: 99%

A Comprehensive Survey on Graph Neural Networks

Pan

Chen

et al. 2021

IEEE Trans. Neural Netw. Learning Syst.

6,146

2,977

View full text Add to dashboard Cite

Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this survey, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art graph neural networks into four categories, namely recurrent graph neural networks, convolutional graph neural networks, graph autoencoders, and spatial-temporal graph neural networks. We further discuss the applications of graph neural networks across various domains and summarize the open source codes, benchmark data sets, and model evaluation of graph neural networks. Finally, we propose potential research directions in this rapidly growing field.

show abstract

“…GNN for NLP: Recently, there is considerable amount of interest in applying GNN to NLP tasks and great success has been achieved. For example, in neural machine translation, GNN has been employed to integrate syntactic and semantic information into encoders (Bastings et al, 2017;Marcheggiani et al, 2018); applied GNN to relation extraction over pruned dependency trees; the study by Yao et al (2018) employed GNN over a heterogeneous graph to do text classification, which inspires our idea of the HDE graph; Liu et al (2018) proposed a new contextualized neural network for sequence learning by leveraging various types of non-local contextual information in the form of information passing over GNN. These studies are related to our work in the sense that we both use GNN to improve the information interaction over long context or across documents.…”

Section: Related Workmentioning

confidence: 99%

Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs

Tao¹,

Wang²,

Huang³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

135

View full text Add to dashboard Cite

Multi-hop reading comprehension (RC) across documents poses new challenge over singledocument RC because it requires reasoning over multiple documents to reach the final answer. In this paper, we propose a new model to tackle the multi-hop RC problem. We introduce a heterogeneous graph with different types of nodes and edges, which is named as Heterogeneous Document-Entity (HDE) graph. The advantage of HDE graph is that it contains different granularity levels of information including candidates, documents and entities in specific document contexts. Our proposed model can do reasoning over the HDE graph with nodes representation initialized with co-attention and self-attention based context encoders. We employ Graph Neural Networks (GNN) based message passing algorithms to accumulate evidences on the proposed HDE graph. Evaluated on the blind test set of the Qangaroo WIKIHOP data set, our HDE graph based single model delivers competitive result, and the ensemble model achieves the state-of-the-art performance.

show abstract

Exploiting Semantics in Neural Machine Translation with Graph Convolutional Networks

Cited by 190 publications

References 19 publications

Semantic Neural Machine Translation Using AMR

Semantic Neural Machine Translation Using AMR

A Comprehensive Survey on Graph Neural Networks

Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs

Contact Info

Product

Resources

About