Graph-based Dependency Parsing with Graph Neural Networks

Ji, Tao; Wu, Yuanbin; Lan, Man

doi:10.18653/v1/p19-1237

Cited by 57 publications

(51 citation statements)

References 29 publications

(37 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, we conduct experiments on Universal Dependencies (UD) v2.2 and v2.3 following Ji et al (2019) and respectively. We adopt the 300d multilingual pretrained word embeddings used in Zeman et al (2018) and take the CharLSTM representations as input.…”

Section: Methodsmentioning

confidence: 99%

Efficient Second-Order TreeCRF for Neural Dependency Parsing

Zhang¹,

Li²,

Zhang³

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

In the deep learning (DL) era, parsing models are extremely simplified with little hurt on performance, thanks to the remarkable capability of multi-layer BiLSTMs in context representation. As the most popular graphbased dependency parser due to its high efficiency and performance, the biaffine parser directly scores single dependencies under the arc-factorization assumption, and adopts a very simple local token-wise cross-entropy training loss. This paper for the first time presents a second-order TreeCRF extension to the biaffine parser. For a long time, the complexity and inefficiency of the inside-outside algorithm hinder the popularity of TreeCRF. To address this issue, we propose an effective way to batchify the inside and Viterbi algorithms for direct large matrix operation on GPUs, and to avoid the complex outside algorithm via efficient back-propagation. Experiments and analysis on 27 datasets from 13 languages clearly show that techniques developed before the DL era, such as structural learning (global TreeCRF loss) and high-order modeling are still useful, and can further boost parsing performance over the state-of-the-art biaffine parser, especially for partially annotated training data. We release our code at https:

show abstract

Section: Methodsmentioning

confidence: 99%

Efficient Second-Order TreeCRF for Neural Dependency Parsing

Zhang¹,

Li²,

Zhang³

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…Equations ( 4) and ( 5) constitute the mechanism by which each iteration of refinement can condition on the previous graph. Instead of the more common approach of hard-coding some attention heads to represent a relation (e.g., Ji et al, 2019), all attention heads can learn for themselves how to use the information about relations.…”

Section: Self-attention Mechanismmentioning

confidence: 99%

Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement

Mohammadshahi

Henderson

2021

Transactions of the Association for Computational Linguistics

View full text Add to dashboard Cite

We propose the Recursive Non-autoregressive Graph-to-Graph Transformer architecture (RNGTr) for the iterative refinement of arbitrary graphs through the recursive application of a non-autoregressive Graph-to-Graph Transformer and apply it to syntactic dependency parsing. We demonstrate the power and effectiveness of RNGTr on several dependency corpora, using a refinement model pre-trained with BERT. We also introduce Syntactic Transformer (SynTr), a non-recursive parser similar to our refinement model. RNGTr can improve the accuracy of a variety of initial parsers on 13 languages from the Universal Dependencies Treebanks, English and Chinese Penn Treebanks, and the German CoNLL2009 corpus, even improving over the new state-of-the-art results achieved by SynTr, significantly improving the state-of-the-art for all corpora tested.

show abstract

“…Akin to graph-based parsers (Ji et al, 2019;Zhang et al, 2019), our model generates parse structures in the form of graphs. In our case, how-ever, graph nodes correspond to syntactic primitives (atomic types & dependencies) rather than words, while the discovery of the graph structure is subject to hard constraints imposed by the decoder's output.…”

Section: Related Workmentioning

confidence: 99%

Neural Proof Nets

Kogkalidis

Moortgat

Moot

2020

Proceedings of the 24th Conference on Computational Natural Language Learning

View full text Add to dashboard Cite

Linear logic and the linear λ-calculus have a long standing tradition in the study of natural language form and meaning. Among the proof calculi of linear logic, proof nets are of particular interest, offering an attractive geometric representation of derivations that is unburdened by the bureaucratic complications of conventional prooftheoretic formats. Building on recent advances in set-theoretic learning, we propose a neural variant of proof nets based on Sinkhorn networks, which allows us to translate parsing as the problem of extracting syntactic primitives and permuting them into alignment. Our methodology induces a batch-efficient, end-to-end differentiable architecture that actualizes a formally grounded yet highly efficient neuro-symbolic parser. We test our approach on AEthel, a dataset of typelogical derivations for written Dutch, where it manages to correctly transcribe raw text sentences into proofs and terms of the linear λcalculus with an accuracy of as high as 70%.

show abstract

Graph-based Dependency Parsing with Graph Neural Networks

Cited by 57 publications

References 29 publications

Efficient Second-Order TreeCRF for Neural Dependency Parsing

Efficient Second-Order TreeCRF for Neural Dependency Parsing

Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement

Neural Proof Nets

Contact Info

Product

Resources

About