A Uniform Method for Automatically Extracting Stochastic Lexicalized Tree Grammars from Treebanks and HPSG

Abstract

We present a uniform method for the extraction of stochastic lexicalized tree grammars (SLTG) of different complexities from existing treebanks as well as from competence-based grammars , which allows us to analyze the relationship of a grammar automatically induced from a treebank with respect to its size, its complexity, and its predictive power on unseen data. Processing of different SLTG is performed by a stochastic version of the two-step Early-based parsing strategy introduced in Schabes and Joshi, 1991.

From Treebank Resources to LFG F-Structures

“…We do believe, however, that the methods can be generalised, and we intend to apply them in an HPSG scenario and to semantic representation based annotations. Our second method could be applied to work in a TAG scenario (see also the closely related work in Neumann(1998 and Neumann and Flickinger(1999)). In our work to date, contrary to an often perceived view, we have found that treebanks encode highly useful linguistic information, albeit often in rather flat representations.…”

A Shallow Text Processing Core Engine

“…Processing of complex phrases and clause patterns will be realized through the compilation of stochastic lexicalized tree grammars (SLTG) into cascades of WFST. An SLTG will be automatically extracted from existing tree banks following our work described in Neumann (1998).…”

A syntactic component for Vietnamese language processing

“…Chen and Vijay-Shanker (2000) and Chen et al (2006) extracted TAGs and there are other works based on Chen's approach such as Johansen (2004) and Nasr (2004) for French, and Habash and Rambow (2004) for Arabic. Neumann (2003) extracted lexicalized tree grammars for English from the English Penn Treebank and for German from the NEGRA treebank. Bäcker and Harbusch (2002) extracted an LTAG grammar for German -also from the NEGRA corpus -and used it for supertagging.…”