Stochastically evaluating the validity of partial parse trees in incremental parsing

Abstract

This paper proposes a method for evaluating the validity of partial parse trees constructed in incremental parsing. Our method is based on stochastic incremental parsing, and it incrementally evaluates the validity for each partial parse tree on a wordby-word basis. In our method, incremental parser returns partial parse trees at the point where the validity for the partial parse tree becomes greater than a threshold. Our technique is effective for improving the accuracy of incremental parsing.

Incremental, Predictive Parsing with Psycholinguistically Motivated Tree-Adjoining Grammar

“…This approach, however, allows multiple unconnected subtrees for a sentence prefix and uses a look-ahead of two words, that is, it does not build connected structures. An example of a TAG parser that is both incremental and builds connected structures is the work of Kato, Matsubara, and Inagaki (2004). This comes at the price of strong simplifying assumptions with respect to the TAG formalism, such as not distinguishing modifiers and arguments.…”

Incremental parsing with monotonic adjoining operation

“…Lombardo and Sturt (1997) and Kato et al (2004) have already adopted this approach. However, their methods have another problem that their adjoining operations cannot preserve lexical dependencies of partial parse trees.…”

“…By using the operation, we can avoid the problem of infinite local ambiguity. This approach has been adopted by Lombardo and Sturt (1997) and Kato et al (2004). However, this raises another problem that their adjoining operations cannot preserve lexical dependencies of partial parse trees.…”

Incremental Parsing with Adjoining Operation

“…By using the operation, we can avoid the problem of infinite local ambiguity. This approach has been adopted by the previous methods [8], [9]. However, this raises another problem that their adjoining operation cannot preserve information which partial parse trees have.…”

“…Lombardo, et al [8] and Kato, et al [9] have already adopted this approach. However, their method causes another problem.…”