Transition-based Neural Constituent Parsing

Abstract

Constituent parsing is typically modeled by a chart-based algorithm under probabilistic context-free grammars or by a transition-based algorithm with rich features. Previous models rely heavily on richer syntactic information through lexicalizing rules, splitting categories, or memorizing long histories. However enriched models incur numerous parameters and sparsity issues, and are insufficient for capturing various syntactic phenomena. We propose a neural network structure that explicitly models the unbounded history of actions performed on the stack and queue employed in transition-based parsing, in addition to the representations of partially parsed tree structure. Our transition-based neural constituent parsing achieves performance comparable to the state-of-the-art parsers, demonstrating F1 score of 90.68% for English and 84.33% for Chinese, without reranking, feature templates or additional data to train model parameters.

Transition-Based Disfluency Detection using LSTMs

“…The effectiveness of neural features has also been studied for this framework Watanabe and Sumita, 2015;Andor et al, 2016). We apply the transition-based neural framework to disfluency detection, which to our knowledge has not been investigated before.…”

End-to-End Neural Relation Extraction with Global Optimization

“…Global optimization has achieved success for several NLP tasks under the neural setting (Zhou et al, 2015;Watanabe and Sumita, 2015). For relation extraction, global learning gives the best performances under the discrete setting (Li and Ji, 2014;Miwa and Sasaki, 2014).…”

“…As has been commonly understood, learning local decisions for structured prediction can lead to label bias (Lafferty et al, 2001), which prevents globally optimal structures from receiving optimal scores by the model. We address this potential issue by building a structural neural model for end-to-end relation extraction, following a recent line of efforts on globally optimized models for neural structured prediction (Zhou et al, 2015;Watanabe and Sumita, 2015;Andor et al, 2016;Wiseman and Rush, 2016).…”

A Neural Probabilistic Structured-Prediction Method for Transition-Based Natural Language Processing

“…We (Zhou et al, 2015) use a globally normalized training objective for modeling the whole action sequence, while Weiss et al only update the parameters of the perceptron layer by employing a perceptronbased objective. Watanabe and Sumita (2015), Xu, Auli, and Clark (2016) and Wiseman and Rush (2016) propose a different structured-prediction neural models with global optimization upon a sequence of actions. Watanabe and Sumita use the neural structured model and beam search on the constituent parsing task.…”