A syntax-based part-of-speech analyser

Abstract

There are two main methodologies for constructing the knowledge base of a natural language analyser: the linguistic and the data-driven. Recent state-of-the-art part-of-speech taggers are based on the data-driven approach. Because of the known feasibility of the linguistic rule-based approach at related levels of description, the success of the data-driven approach in part-of-speech analysis may appear surprising. In this paper 1 , a case is made for the syntactic nature of part-of-speech tagging. A new tagger of English that uses only linguistic distributional rules is outlined and empirically evaluated. Tested against a benchmark corpus of 38,000 words of previously unseen text, this syntax-based system reaches an accuracy of above 99%. Compared to the 95-97% accuracy of its best competitors, this result suggests the feasibility of the linguistic approach also in part-of-speech analysis.

La Recuperación de Información en español y la normalizacion de términos

“…para el Tagger ENGCG se ha estimado una corrección en el etiquetado del 99.5 %, aunque dejando un 3.2 % de palabras sin desambiguar [41]-, hemos optado por construir un desambiguador basado en los modelos estadísti-cos, pues el coste de desarrollo es claramente inferior al necesario para la implementación de uno basado en reglas [34].…”

Shallow language processing architecture for Bulgarian

“…Since large tagged corpora in Bulgarian are not widely available, the development of a corpus-based probabilistic tagger was an unrealistic goal for us. However, as some studies suggest (Voutilainen, 1995), the precision of rule-based taggers may exceed that of the probabilistic ones.…”

Inducing constraint grammars

“…EngCG is reported 11,9,7,8] to assign a correct analysis to about 99.7 % of all words; on the other hand, each word retains on average 1.04{1.09 alternative analyses, i.e. some of the ambiguities remain unresolved.…”