Can you tag the modal? You should

Abstract

Computational linguistics methods are typically first developed and tested in English. When applied to other languages, assumptions from English data are often applied to the target language. One of the most common such assumptions is that a "standard" part-of-speech (POS) tagset can be used across languages with only slight variations. We discuss in this paper a specific issue related to the definition of a POS tagset for Modern Hebrew, as an example to clarify the method through which such variations can be defined. It is widely assumed that Hebrew has no syntactic category of modals. There is, however, an identified class of words which are modal-like in their semantics, and can be characterized through distinct syntactic and morphologic criteria. We have found wide disagreement among traditional dictionaries on the POS tag attributed to such words. We describe three main approaches when deciding how to tag such words in Hebrew. We illustrate the impact of selecting each of these approaches on agreement among human taggers, and on the accuracy of automatic POS taggers induced for each method. We finally recommend the use of a "modal" tag in Hebrew and provide detailed guidelines for this tag. Our overall conclusion is that tagset definition is a complex task which deserves appropriate methodology.

Word Segmentation, Unknown-word Resolution, and Morphological Agreement in a Hebrew Parsing System

“…These are not mere technical differences, but derive from different perspectives on the data. The Hebrew Treebank (TB) tag set is syntactic in nature ("if the word in this particular position functions as an adverb, tag it as an adverb, even though it is listed in the dictionary only as a noun"), whereas the KC tag set (Adler 2007;Netzer et al 2007;Adler et al 2008b) takes a lexical approach to POS tagging ("a word can assume only POS tags that would be assigned to it in a dictionary"). The lexical approach does not accommodate generic modification POS tags such as MOD, nor does it allow listing of demonstrative pronouns as adjectives.…”

“…This kind of disagreement naturally appears also between the KC and TB. See Adler et al (2008b) and Netzer et al (2007) for further discussion on these two interesting cases.…”

“…In fact, the nine modals are very similar in characterization to the words identified inNetzer et al (2007) as modals, whereas many of the modals in the other group are not necessarily considered as modal outside of the treebank guidelines.…”

Lexical Opposition in Discourse Contrast

“…pers., pron. suff., punt., sost., vb. One could also envisage the refining of the tagset by adding: interrogative, modal, negation, and quantifier (Adler, 2007) (Netzer and Elhadad, 1998) (Netzer et al, 2007).…”

Lexical Opposition in Discourse Contrast

“…pers., pron. suf., punt., sost., vb. One could also envisage the reining of the tagset by adding: interrogative, modal, negation, and quantiier (Adler, 2007) (Netzer and Elhadad, 1998) (Netzer et al, 2007).…”