Dependency Grammar Induction with a Neural Variational Transition-Based Parser

Li, Bowen; Cheng, Jianpeng; Liu, Yang; Keller, Frank

doi:10.1609/aaai.v33i01.33016658

Cited by 23 publications

(34 citation statements)

References 22 publications

(34 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Williams et al (2018) observe a similar phenomenon in the context of learning latent trees for classification tasks. However Li et al (2019) find that it is possible use a transition-based parser as the inference network for dependency grammar induction, if the inference network is constrained via posterior regularization (Ganchev et al, 2010) based on universal syntactic rules (Naseem et al, 2010).…”

Section: Amortized Variational Inferencementioning

confidence: 99%

Unsupervised Recurrent Neural Network Grammars

Kim¹,

Rushton²,

Yu³

et al. 2019

Proceedings of the 2019 Conference of the North

100

109

View full text Add to dashboard Cite

Recurrent neural network grammars (RNNG) are generative models of language which jointly model syntax and surface structure by incrementally generating a syntax tree and sentence in a top-down, left-to-right order. Supervised RNNGs achieve strong language modeling and parsing performance, but require an annotated corpus of parse trees. In this work, we experiment with unsupervised learning of RNNGs. Since directly marginalizing over the space of latent trees is intractable, we instead apply amortized variational inference. To maximize the evidence lower bound, we develop an inference network parameterized as a neural CRF constituency parser. On language modeling, unsupervised RNNGs perform as well their supervised counterparts on benchmarks in English and Chinese. On constituency grammar induction, they are competitive with recent neural language models that induce tree structures from words through attention mechanisms.

show abstract

Section: Amortized Variational Inferencementioning

confidence: 99%

Unsupervised Recurrent Neural Network Grammars

Kim¹,

Rushton²,

Yu³

et al. 2019

Proceedings of the 2019 Conference of the North

100

109

View full text Add to dashboard Cite

show abstract

“…Furthermore, we compare our model with current state-of-the-art discriminative models, the neural variational transition-based parser (NVTP) (Li et al, 2019) (Noji et al, 2016). DV: deterministic variant of D-NDMV.…”

Section: Results On Universal Dependency Treebankmentioning

confidence: 99%

“…VV: variational variant of D-NDMV. NVTP: neural variational transition-based parser (Li et al, 2019). CM: Convex-MST.…”

Section: Results On Universal Dependency Treebankmentioning

confidence: 99%

“…More recently, researchers have started to utilize discriminative methods in unsupervised dependency parsing based on the idea of discriminative clustering (Grave and Elhadad, 2015), the CRFAE framework (Cai et al, 2017) or the neural variational transition-based parser (Li et al, 2019). By conditioning dependency prediction on the whole input sentence, discriminative methods are capable of utilizing not only local information, but also global and contextual information of a dependency in determining its strength.…”

Section: Introductionmentioning

confidence: 99%

“…In this way, they break the context-free independence assumption because the same dependency would have different strength in different contexts. Besides, Li et al (2019) propose a variational autoencoder approach based on Recurrent Neural Network Grammars.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Enhancing Unsupervised Generative Dependency Parser with Contextual Information

Han

Jiang

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Most of the unsupervised dependency parsers are based on probabilistic generative models that learn the joint distribution of the given sentence and its parse. Probabilistic generative models usually explicit decompose the desired dependency tree into factorized grammar rules, which lack the global features of the entire sentence. In this paper, we propose a novel probabilistic model called discriminative neural dependency model with valence (D-NDMV) that generates a sentence and its parse from a continuous latent representation, which encodes global contextual information of the generated sentence. We propose two approaches to model the latent representation: the first deterministically summarizes the representation from the sentence and the second probabilistically models the representation conditioned on the sentence. Our approach can be regarded as a new type of autoencoder model to unsupervised dependency parsing that combines the benefits of both generative and discriminative techniques. In particular, our approach breaks the context-free independence assumption in previous generative approaches and therefore becomes more expressive. Our extensive experimental results on seventeen datasets from various sources show that our approach achieves competitive accuracy compared with both generative and discriminative state-of-the-art unsupervised dependency parsers.

show abstract