Gut, Besser, Chunker – Selecting the Best Models for Text Chunking with Voting

Abstract

The CoNLL-2000 dataset is the de-facto standard dataset for measuring chunkers on the task of chunking base noun phrases (NP) and arbitrary phrases. The state-of-the-art tagging method is utilising TnT, an HMM-based Part-of-Speech tagger (POS), with simple majority voting on different representations and fine-grained classes created by lexcialising tags. In this paper the state-of-the-art English phrase chunking method was deeply investigated, re-implemented and evaluated with several modifications. We also investigated a less studied side of phrase chunking, i.e. the voting between different currently available taggers, the checking of invalid sequences and the way how the state-of-the-art method can be adapted to morphologically rich, agglutinative languages. We propose a new, mild level of lexicalisation and a better combination of representations and taggers for English. The final architecture outperformed the state-of-the-art for arbitrary phrase identification and NP chunking achieving the F-score of 95.06% for arbitrary and 96.49% for noun phrase chunking.

Less is More, More or Less... Finding the Optimal Threshold for Lexicalization in Chunking

“…The lexicalization invented by Molina and Pla [4] is thoroughly investigated by Indig and Endrédy [2] and they present a mildly lexicalized variant (see Table 2) of the method that has superior performance.…”

“…The latter problem can be overcome by using a proper IOB converter which can also fix well-formedness issues (see Section 2 for details). The main motivation of the approach of Indig and Endrédy [2] was to reduce the number of labels because for agglutinative languages the original method is not feasible due to the high number of tags even without any lexicalization. But we also remark that this problem also exists with low thresholds used during lexicalization.…”

“…The CoNLL-2000 dataset [7] is the de facto standard dataset for measuring tagger performance in chunking for English. The current state-of-the-art method Gut, Besser, Chunker [2] achieves the F-score of 95.06% (96.49% for NPs).…”

“…Indig and Endrédy questioned the methodology behind SS05 and compared different taggers on different levels of lexicalization with different voting schemes [2]. The authors could not reproduce the previous results due to the many programming errors discovered in the original code.…”

“…We evaluate all five previous representations, even though it was shown that IOBES has superior performance to the other four [2].…”