Improvements on a semi-automatic grammar induction framework

Abstract

This work extends the semi-automatic grammar induction approach previously proposed in [I]. The data-driven approach leams semantic and phrasal categories h m a mining corpus of unnnnolated natural language queries in a specific domain. The approach can be seeded with pre-specified semantic categories to expedite the leaming process. Grammar rules areautomatically acquired by an agglomerative clustering procedure, and the resulting grammar may be hand-edited easily for refmement. 7his work attempts to improve the grammar induction framework by leveraging information in the SQL query that accompanies every mining query. The SQL expression specifies the action of database access in relation to the query, and hence provides information about meaningful natural language strucNres that should to be captured in induced grammar. We have also incorporated the use of Information Gain in place of Mulual Information to capme phmsal structures, as well as the determination of an automatic stopping criterion for agglomerative clustering.