Viterbi training in PRISM

Abstract

VT (Viterbi training), or hard EM, is an efficient way of parameter learning for probabilistic models with hidden variables. Given an observation $y$, it searches for a state of hidden variables $x$ that maximizes $p(x,y \mid \theta)$ by coordinate ascent on parameters $\theta$ and $x$. In this paper we introduce VT to PRISM, a logic-based probabilistic modeling system for generative models. VT improves PRISM in three ways. First VT in PRISM converges faster than EM in PRISM due to the VT's termination condition. Second, parameters learned by VT often show good prediction performance compared to those learned by EM. We conducted two parsing experiments with probabilistic grammars while learning parameters by a variety of inference methods, i.e.\ VT, EM, MAP and VB. The result is that VT achieved the best parsing accuracy among them in both experiments. Also we conducted a similar experiment for classification tasks where a hidden variable is not a prediction target unlike probabilistic grammars. We found that in such a case VT does not necessarily yield superior performance. Third since VT always deals with a single probability of a single explanation, Viterbi explanation, the exclusiveness condition that is imposed on PRISM programs is no more required if we learn parameters by VT. Last but not least we can say that as VT in PRISM is general and applicable to any PRISM program, it largely reduces the need for the user to develop a specific VT algorithm for a specific model. Furthermore since VT in PRISM can be used just by setting a PRISM flag appropriately, it makes VT easily accessible to (probabilistic) logic programmers. To appear in Theory and Practice of Logic Programming (TPLP).Comment: 23 pages, 1 figur

Lifted discriminative learning of probabilistic logic programs

“…Probabilistic Logic Programming (PLP) is a powerful tool for reasoning in uncertain relational domains that is gaining popularity in Statistical Relational Artificial Intelligence (StarAI) due to its expressiveness and intuitiveness. PLP has been applied successfully to a variety of fields, such as natural language processing (Sato and Kubota 2015;Riguzzi et al 2017b;Nguembang Fadja and Riguzzi 2017), bioinformatics (Mørk and Holmes 2012;De Raedt et al 2007;Sato and Kameya 1997), link prediction in social networks (Meert et al 2010), entity resolution (Riguzzi 2014) and model checking (Gorlin et al 2012).…”

Probabilistic Logic Programming in Action

“…PLP has been applied successfully to many problems such as concept relatedness in biological networks [12], Mendel's genetic inheritance [50], natural language processing [51,44], link prediction in social networks [24], entity resolution [37] and model checking [15].…”