A generative re-ranking model for dependency parsing

Abstract

We propose a framework for dependency parsing based on a combination of discriminative and generative models. We use a discriminative model to obtain a kbest list of candidate parses, and subsequently rerank those candidates using a generative model. We show how this approach allows us to evaluate a variety of generative models, without needing different parser implementations. Moreover, we present empirical results that show a small improvement over state-of-the-art dependency parsing of English sentences.

The Inside-Outside Recursive Neural Network model for Dependency Parsing

“…The third-order model we suggest is similar to the grandsibling model proposed by Sangati et al (2009) and Hayashi et al (2011). It defines the probability of generating a dependent D = dist, d, w, c, t as the product of the distancebased probability and the probabilities of generating each of its components (d, t, w, c, denoting dependency relation, POS-tag, word and capitalisation feature, respectively).…”

“…which is identical to Sangati et al (2009). Moreover, as in many mixture-model-based approaches, we define the mixture reranker as a combination of the generative model and the MST discriminative model (Hayashi et al, 2011)…”

“…Hayashi et al (2011) then followed this idea but combined generative models with a variational model learnt on the fly to rerank forests. In this paper, we also followed Sangati et al (2009)'s idea but used an ∞-order generative model, which has never been used before.…”

Dependency Parse Reranking with Rich Subtree Features

“…"Hall07" stands for a Maximum Entropy k-best reranker [6]. "Sangati09" stands for a discriminative k-best parser pipelined with a generative reranker [20]. "Hayashi11" is a forest reranker based on a variant of Eisner's generative model [7].…”

“…Huang and Chiang proposed an efficient algorithm for producing the K-best list for graph-based parsers, which add a factor of to the parsing complexity of the base parser [5]. Sangati et al has shown that a discriminative parser is very effective at filtering out bad parses from a factorized search space [20]. This agreed with the conclusion of Hall [6], who found that an edge-factored model can attain good oracle performance when generating a relatively small K-best list.…”