An Effective Neural Network Model for Graph-based Dependency Parsing

Abstract

Most existing graph-based parsing models rely on millions of hand-crafted features, which limits their generalization ability and slows down the parsing speed. In this paper, we propose a general and effective Neural Network model for graph-based dependency parsing. Our model can automatically learn high-order feature combinations using only atomic features by exploiting a novel activation function tanhcube. Moreover, we propose a simple yet effective way to utilize phrase-level information that is expensive to use in conventional graph-based parsers. Experiments on the English Penn Treebank show that parsers based on our model perform better than conventional graph-based parsers.

Graph-based Dependency Parsing with Bidirectional LSTM

“…Contextual information of word pairs 1 has been widely utilized in previous work (McDonald et al, 2005;McDonald and Pereira, 2006;Pei et al, 2015). For a dependency pair (h, m), previous work divides a sentence into three parts (prefix, infix and suffix) by head word h and modifier word m. These parts which we call segments in our work make up the context of the dependency pair (h, m).…”

“…The neural network model proposed by Pei et al (2015) alleviates the dependence on feature engineering to a large extent, but not completely. We follow Pei et al (2015) to score dependency arcs using neural network model.…”

“…The neural network model proposed by Pei et al (2015) alleviates the dependence on feature engineering to a large extent, but not completely. We follow Pei et al (2015) to score dependency arcs using neural network model. However, different from their work, we introduce a Bidirectional LSTM to capture long range contextual information and an extra forward LSTM to better represent segments of the sentence separated by the head and modifier.…”

“…To alleviate the burden of feature engineering, Pei et al (2015) presented an effective neural network model for graph-based dependency parsing. They only use atomic features such as word unigrams and POS tag unigrams and leave the model to automatically learn the feature combinations.…”

Semi-supervised Dependency Parsing using Bilexical Contextual Features from Auto-Parsed Data

“…Recently, progress in dependency parsing has been made by introducing non-linear, neuralnetwork based models (Pei et al, 2015;Chen and Manning, 2014;Weiss et al, 2015;Dyer et al, 2015;Zhou et al, 2015). Adapting our approach to work with such models is an interesting research direction.…”