Semi-supervised Dependency Parsing using Bilexical Contextual Features from Auto-Parsed Data

Abstract

We present a semi-supervised approach to improve dependency parsing accuracy by using bilexical statistics derived from auto-parsed data. The method is based on estimating the attachment potential of head-modifier words, by taking into account not only the head and modifier words themselves, but also the words surrounding the head and the modifier. When integrating the learned statistics as features in a graph-based parsing model, we observe nice improvements in accuracy when parsing various English datasets.

Association Metrics in Neural Transition-Based Dependency Parsing

“…In parsing, bilexical preferences have been used by Van Noord (2007) to improve syntactic ambiguity resolution in a Maximum-Entropy parser for Dutch. Kiperwasser and Goldberg (2015) extended bilexical preferences to contextual association scores based on PMI and dependency embeddings (Levy and Goldberg, 2014a) in a graph-based parser. Mirroshandel and Nasr (2016) integrated selectional preferences into a graph-based dependency parser.…”

Delexicalized Word Embeddings for Cross-lingual Dependency Parsing

“…In these approaches, word co-occurrences are defined in terms of dependency contexts (x is the governor of word w), instead of linear contexts (x appears within a range of s around word w). Embedding techniques have also started to be applied to objects other than words, namely on dependency relations (Bansal, 2015;Kiperwasser and Goldberg, 2015).…”

Expected F-Measure Training for Shift-Reduce Parsing with Recurrent Neural Networks

“…A line of work is devoted to parsing with RNN models, including using RNNs (Miikkulainen, 1996;Mayberry and Miikkulainen, 1999;Legrand and Collobert, 2015;Watanabe and Sumita, 2015) and LSTM (Hochreiter and Schmidhuber, 1997) RNNs Kiperwasser and Goldberg, 2016). Legrand and Collobert (2015) used RNNs to learn conditional distributions over syntactic rules; explored sequenceto-sequence learning (Sutskever et al, 2014) for parsing; utilized characterlevel representations and Kiperwasser and Goldberg (2016) built an easy-first dependency parser using tree-structured compositional LSTMs. However, all these parsers use greedy search and are trained using the maximum likelihood criterion (except Kiperwasser and Goldberg (2016), who used a margin-based objective).…”

“…Legrand and Collobert (2015) used RNNs to learn conditional distributions over syntactic rules; explored sequenceto-sequence learning (Sutskever et al, 2014) for parsing; utilized characterlevel representations and Kiperwasser and Goldberg (2016) built an easy-first dependency parser using tree-structured compositional LSTMs. However, all these parsers use greedy search and are trained using the maximum likelihood criterion (except Kiperwasser and Goldberg (2016), who used a margin-based objective). For learning global models, Watanabe and Sumita (2015) used a marginbased objective, which was not optimized for the evaluation metric; although not using RNNs, Weiss et al (2015) proposed a method using the averaged perceptron with beam search (Collins, 2002;Collins and Roark, 2004;Zhang and Clark, 2008), which required fixing the neural network representations, and thus their model parameters were not learned using end-to-end backpropagation.…”