EM-Training for Weighted Aligned Hypergraph Bimorphisms

Abstract

We develop the concept of weighted aligned hypergraph bimorphism where the weights may, in particular, represent probabilities. Such a bimorphism consists of an R ≥0 -weighted regular tree grammar, two hypergraph algebras that interpret the generated trees, and a family of alignments between the two interpretations. Semantically, this yields a set of bihypergraphs each consisting of two hypergraphs and an explicit alignment between them; e.g., discontinuous phrase structures and nonprojective dependency structures are bihypergraphs. We present an EM-training algorithm which takes a corpus of bihypergraphs and an aligned hypergraph bimorphism as input and generates a sequence of weight assignments which converges to a local maximum or saddle point of the likelihood function of the corpus.

Weighted parsing for grammar-based language models

“…We mention that also context-free hypergraph grammars [BC87; HK87] can be viewed as RTG-LMs [Cou91] (also cf. [DGV16]). Each of these classes is determined by a particular class of language algebras.…”

“…[NS03]). Drewes, Gebhardt, and Vogler [DGV16] generalized the use of the intersection in EM training to language models beyond CFG.…”

“…For this, we employ weighted regular tree grammars (wRTG) and evaluate the generated trees (by the unique homomorphism) in some language algebra L, which provides the set of syntactic objects as carrier set. This approach is very flexible and covers LMs for strings (CFG, LCFRS), but also trees and graphs (Drewes et al, 2016). Our second generalization concerns the weight algebra.…”