Syntax Role for Neural Semantic Role Labeling

Abstract

Semantic role labeling (SRL) is dedicated to recognizing the semantic predicate-argument structure of a sentence. Previous studies in terms of traditional models have shown syntactic information can make remarkable contributions to SRL performance; however, the necessity of syntactic information was challenged by a few recent neural SRL studies that demonstrate impressive performance without syntactic backbones and suggest that syntax information becomes much less important for neural semantic role labeling, especially when paired with recent deep neural network and large-scale pre-trained language models. Despite this notion, the neural SRL field still lacks a systematic and full investigation on the relevance of syntactic information in SRL, for both dependency and both monolingual and multilingual settings. This paper intends to quantify the importance of syntactic information for neural SRL in the deep learning framework. We introduce three typical SRL frameworks (baselines), sequence-based, tree-based, and graph-based, which are accompanied by two categories of exploiting syntactic information: syntax pruningbased and syntax feature-based. Experiments are conducted on the CoNLL-2005, 2009, and 2012 benchmarks for all languages available, and results show that neural SRL models can still benefit from syntactic information under certain conditions. Furthermore, we show the quantitative significance of syntax to neural SRL models together with a thorough empirical survey using existing models.

Transition-based Semantic Role Labeling with Pointer Networks

“…It can be seen as predicate-centered SRL systems outnumber those designed for full SRL (which are reported in Table 3). While our approach was originally designed for dealing with the lack of gold predicates and just minor adaptations were undertaken for leveraging pre-identified predicate information, our system behaves similarly to the full SRL setting: it is the best-performing syntax-agnostic approach without contextualized word representations (also surpassing syntaxaware systems on the WSJ test set), but the highest scores are reported by graph-based models that fine-tune BERTbased embeddings (Li et al, 2020) or, while keeping them frozen, leverage syntactic information (Li et al, 2021).…”

“…Syntax-based approaches (Roth and Lapata, 2016;He et al, 2018He et al, , 2019Cai and Lapata, 2019b;Li et al, 2021) have been the mainstream for dependency-based SRL, consistently proving that syntactic information is highly effective for achieving state-of-the-art accuracies.…”

“…However, the wide majority of these end-to-end approaches do not perform predicate identification and, therefore, are not considered full SRL systems. Moreover, most previous efforts mainly focused on syntax-aware SRL methods (He et al, 2018(He et al, , 2019Cai and Lapata, 2019b;Kasai, Friedman, Frank, Radev and Rambow, 2019;Li, Zhao, He and Cai, 2021): i.e., they leverage syntactic information (also provided by the CoNLL-2009 corpora) to produce state-of-the-art accuracies, motivated, among other reasons, by the fact that a significant portion of arcs from syntactic dependency trees matches predicateargument relations in SRL structures. Nevertheless, it is important to note that dependency parsing is also an equally challenging and resource-consuming NLP task, and syntactic training data can be especially scarce in low-resource languages.…”

Seeking Common but Distinguishing Difference, A Joint Aspect-based Sentiment Analysis Model

“…Recently, NLP has been developed rapidly Li et al, 2019b;Jiang et al, 2020;, and the process is further by deep neural networks (Parnow et al, 2021;Li et al, 2021a) and pre-trained language models (Li et al, 2021b;Zhang et al, 2020b). Aspect-based sentiment analysis was proposed by Pontiki et al (2014) and also received lots of attention in recent years.…”

Semantic Role Labeling for Amharic Text Using Multiple Embeddings and Deep Neural Network

“…SRL systems using deep learning can, therefore, easily learn important representations from just the raw input and can avoid the need to address the SRL problem in modular steps such as prunning, argument identification and argument classification. From the perspective of implementation, neural-based semantic role labeling systems can be modeled using dependency-based style [14], [15], spanbased style [16], [17], or cross-based style [18]. Dependencybased and span-based styles annotate the syntactic heads of arguments and the entire argument span, respectively whereas cross-based style integrates both.…”