Detecting negation scope is easy, except when it isn't

Abstract

Several corpora have been annotated with negation scope-the set of words whose meaning is negated by a cue like the word "not"-leading to the development of classifiers that detect negation scope with high accuracy. We show that for nearly all of these corpora, this high accuracy can be attributed to a single fact: they frequently annotate negation scope as a single span of text delimited by punctuation. For negation scopes not of this form, detection accuracy is low and undersampling the easy training examples does not substantially improve accuracy. We demonstrate that this is partly an artifact of annotation guidelines, and we argue that future negation scope annotation efforts should focus on these more difficult cases.

Neural Networks For Negation Scope Detection

“…Modeling syntax in addition to surface word order is helpful, as shown by an ensemble of BiLSTM and D-LSTM models outperforming either model alone. Our results also show that cross-lingual word embeddings are not really necessary, suggesting that the model mainly relies on PoS, syntax, and punctuation boundaries-with the latter result reinforcing previous findings (Fancellu et al, 2017).…”

“…Modeling syntax is useful, though not on its own. The ensembles that incorporate syntax outperform other models on both F 1 and PCS in both the monolingual and crosslingual settings, showing that syntax is indeed beneficial-note that they outpeform the stateof-the-art BiLSTM of Fancellu et al (2016Fancellu et al ( , 2017. 4 The D-LSTM outperforms the GCN in the monolingual settings but the latter performs better in terms of full scope spans detected in the when training in English and testing in Chinese.…”

“…This might be an artifact of the two languages used in the experiment, since English and Chinese have similar word order. However, this also could be explained by a striking observation by Fancellu et al (2017), who showed that recurrent classifiers are very accurate when negation scope is delimited by punctuation and sentence boundaries but inaccurate otherwise. For example, they would correctly predict the scope in Ex.…”

“…First, we replicate the experiments Fancellu et al (2017) and divide the development instances into two groups, the easy instances, predictable by punctuation alone and the hard instances where scope cannot be predicted by punctuation alone. If the predictions of the BiLSTM are guided by punctuation we would expect easy instances to be predicted correctly more often than hard ones.…”

Neural Token Representations and Negation and Speculation Scope Detection in Biomedical and General Domain Text

“…• Following Fancellu et al (2017), we provide a thorough comparison of our proposed model with other state-of-the-art models and analyze their behaviour in the absence of potential "linear clues", the presence of which might result in highly accurate predictions even for syntax-unaware token representations.…”