Conditional Density Approximations with Mixtures of Polynomials

Abstract

Mixtures of polynomials (MoPs) are a nonparametric density estimation technique especially designed for hybrid Bayesian networks with continuous and discrete variables. Algorithms to learn one-and multidimensional (marginal) MoPs from data have recently been proposed. In this paper, we introduce two methods for learning MoP approximations of conditional densities from data. Both approaches are based on learning MoP approximations of the joint density and the marginal density of the conditioning variables, but they differ as to how the MoP approximation of the quotient of the two densities is found. We illustrate and study the methods using data sampled from known parametric distributions, and demonstrate their applicability by learning models based on real neuroscience data. Finally, we compare the performance of the proposed methods with an approach for learning mixtures of truncated basis functions (MoTBFs). The empirical results show that the proposed methods generally yield models that are comparable to or significantly better than those found using the MoTBF-based method. C 2014 Wiley Periodicals, Inc.

Learning Conditional Distributions Using Mixtures of Truncated Basis Functions

“…The results in Table 1 indicate that the most accurate results for scenario 1 are achieved by the B-spline approach [11]. The worst results by far are obtained by the approach that discretizes the conditioning variables [5].…”

“…Here, however, we will follow an alternative strategy similar to the one pursued in [11]. The idea is to learn representations for f (x, z) and f (z), then utilize Equation 4 to calculate f (x|z).…”

“…As already noted, this will not result in an MoP, and the next step is therefore to approximate this representation into an MoP by some means. Varando et al [11] investigated two schemes: i) To use the representation in Equation 4 to generate samples from the conditional distribution of x given Z and learn the MoP representation from the generated dataset; ii) to use numerical techniques to approximate the fraction directly (specifically, both Taylor series and Lagrange interpolation were considered). In our work we first learn an MoP representation for f (x, z) using the program in Equation 3, then calculate f (z) = Ωx f (x, z)dx directly from the learned joint.…”

“…In this section, we compare the proposal given in Section 3 with the methods described in [5] (where the conditioning variables are discretized) and in [11] (where B-splines are used) for learning conditional MoPs from data.…”

“…However, even though a Bayesian network model populated with MoTBF distributions requires the specification of both marginal and conditional MoTBF distributions, only limited attention has been given to learning the conditional MoTBF distributions directly from data [1,11]. In this paper we first extend previous work on learning marginal MoTBF distributions [5] to also learn joint densities.…”