Sequence Labeling Parsing by Learning across Representations

Abstract

We use parsing as sequence labeling as a common framework to learn across constituency and dependency syntactic abstractions. To do so, we cast the problem as multitask learning (MTL). First, we show that adding a parsing paradigm as an auxiliary loss consistently improves the performance on the other paradigm. Secondly, we explore an MTL sequence labeling model that parses both representations, at almost no cost in terms of performance and speed. The results across the board show that on average MTL models with auxiliary losses for constituency parsing outperform singletask ones by 1.05 F1 points, and for dependency parsing by 0.62 UAS points.

Sequence Tagging for Fast Dependency Parsing

“…Finally, the best result for dependency parsing is achieved when adding constituency parsing as auxiliary task (D-MTL-AUX). More experiments on various languages and the reported speeds when including the MTL approach are presented in [12].…”

Mimic and Conquer: Heterogeneous Tree Structure Distillation for Syntactic NLP

“…Generally, these methods consider injecting either standalone constituency tree or dependency tree by tree encoders such as TreeL-STM (Socher et al, 2013;Tai et al, 2015a) or GCN (Kipf and Welling, 2017). Based on the assumption that the dependency and constituency representation can be disentangled and coexist in one shared model, existing efforts are paid for joint constituent and dependency parsing, verifying the mutual benefit of these heterogeneous structures (Collins, 1997;Charniak, 2000;Charniak and Johnson, 2005;Farkas et al, 2011;Ren et al, 2013;Yoshikawa et al, 2017;Strzyz et al, 2019;Kato and Matsubara, 2019;Zhou and Zhao, 2019). However, little attention is paid for facilitating the syntax-dependent tasks via integrating heterogeneous syntactic trees.…”

A hybrid representation-based simile component extraction

“…Rei [28] considers that these patterns are useful for improving accuracy on sequence labeling tasks. Strzyz et al [32] use sequence labeling for constituency [11] and dependency parsing [33] combined with multi-task learning to learn across syntactic representations. They show that adding a parsing paradigm as an auxiliary loss consistently improves the performance on the other paradigm.…”

“…What's more, they further demonstrate that a single multi-task learning model following their strategy can robustly produce constituency and dependency trees. This model obtains a performance and speed comparable with previous sequence labeling models for constituency and dependency parsing [32]. Chen and Moschitti [7] propose an approach for transferring the knowledge of a neural model for sequence labeling, learned from the source domain, to a new model trained on a target domain, where new label categories appear.…”