Marginal Pseudo-Likelihood Learning of Discrete Markov Network Structures

Pensar, Johan; Nyman, Henrik; Niiranen, Juha; Corander, Jukka

doi:10.1214/16-ba1032

Cited by 24 publications

(65 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The main contribution of this work is extending the scope of marginal pseudo‐likelihood (MPL) to also cover CMNs. This was achieved by combining the concept of marginal pseudo‐likelihood for Markov networks (Pensar et al, ) with the concept of local CSIs in Bayesian network learning (Boutilier et al, ; Friedman & Goldszmidt, ; Pensar et al, ). The resulting objective function has a tractable closed‐form expression and the corresponding estimator was here proven to be consistent in the large sample limit.…”

Section: Discussionmentioning

confidence: 99%

“…Proof. (Theorem 1) Pensar et al (2016b) showed that MPL is consistent in identifying the correct graph for traditional MNs. Considering maximal regular structures, the MPL is also consistent in identifying the correct graph of a CMN.…”

Section: Appendixmentioning

confidence: 96%

“…The MPL is now obtained by replacing the likelihood function in with the pseudo‐likelihood function. Under certain assumptions regarding parameter independence (Pensar et al ,), the parameter prior can be factorized in a similar fashion. By further assuming

(θ_{1 j l}, \dots, θ_{r_{j} j l}) \sim Dirichlet (α_{1 j l}, \dots, α_{r_{j} j l}) for all j = 1, \dots, d and l = 1, \dots, q_{j},

the MPL can be evaluated by the closed‐form expression

\overset{p}{true} (boldx | G) = {falsefalse}_{\prod}^{j = 1} {falsefalse}_{\prod}^{l = 1} \frac{normal Γ (α_{j l})}{normal Γ (n_{j l} + α_{j l})} {falsefalse}_{\prod}^{i = 1} \frac{normal Γ (n_{i j l} + α_{i j l})}{normal Γ (α_{i j l})}

where

n_{j l} = \sum_{i = 1}^{r_{j}} n_{i j l}

and

α_{j l} = \sum_{i = 1}^{r_{j}} α_{i j l}

.…”

Section: Marginal Pseudo‐likelihood Learning Of Contextual Markov Netmentioning

confidence: 99%

“…It has been shown that the MPL criterion is a consistent estimator for the graph structure (Pensar et al, ). In the following theorem, we establish that the modified MPL criterion is also a consistent estimator of the dependence structure of CMNs.…”

Section: Marginal Pseudo‐likelihood Learning Of Contextual Markov Netmentioning

confidence: 99%

“…where the l-index indicates the q j distinct classes specified by C.j /. By using Stirling's asymptotic formula and following the same approach as in Pensar et al (2016b), we obtain the asymptotic result…”

Section: Appendixmentioning

confidence: 99%

See 4 more Smart Citations

Structure Learning of Contextual Markov Networks using Marginal Pseudo‐likelihood

Pensar¹,

Nyman²,

Corander

2017

Scandinavian J Statistics

Self Cite

View full text Add to dashboard Cite

Markov networks are popular models for discrete multivariate systems where the dependence structure of the variables is specified by an undirected graph. To allow for more expressive dependence structures, several generalizations of Markov networks have been proposed. Here, we consider the class of contextual Markov networks which takes into account possible context‐specific independences among pairs of variables. Structure learning of contextual Markov networks is very challenging due to the extremely large number of possible structures. One of the main challenges has been to design a score, by which a structure can be assessed in terms of model fit related to complexity, without assuming chordality. Here, we introduce the marginal pseudo‐likelihood as an analytically tractable criterion for general contextual Markov networks. Our criterion is shown to yield a consistent structure estimator. Experiments demonstrate the favourable properties of our method in terms of predictive accuracy of the inferred models.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Appendixmentioning

confidence: 96%

(θ_{1 j l}, \dots, θ_{r_{j} j l}) \sim Dirichlet (α_{1 j l}, \dots, α_{r_{j} j l}) for all j = 1, \dots, d and l = 1, \dots, q_{j},

the MPL can be evaluated by the closed‐form expression

\overset{p}{true} (boldx | G) = {falsefalse}_{\prod}^{j = 1} {falsefalse}_{\prod}^{l = 1} \frac{normal Γ (α_{j l})}{normal Γ (n_{j l} + α_{j l})} {falsefalse}_{\prod}^{i = 1} \frac{normal Γ (n_{i j l} + α_{i j l})}{normal Γ (α_{i j l})}

where

n_{j l} = \sum_{i = 1}^{r_{j}} n_{i j l}

and

α_{j l} = \sum_{i = 1}^{r_{j}} α_{i j l}

.…”

Section: Marginal Pseudo‐likelihood Learning Of Contextual Markov Netmentioning

confidence: 99%

Section: Marginal Pseudo‐likelihood Learning Of Contextual Markov Netmentioning

confidence: 99%

Section: Appendixmentioning

confidence: 99%

See 3 more Smart Citations

Structure Learning of Contextual Markov Networks using Marginal Pseudo‐likelihood

Pensar¹,

Nyman²,

Corander

2017

Scandinavian J Statistics

Self Cite

View full text Add to dashboard Cite

show abstract

High-dimensional structure learning of sparse vector autoregressive models using fractional marginal pseudo-likelihood

Suotsalo

Corander

et al. 2021

Stat Comput

Self Cite

View full text Add to dashboard Cite

Learning vector autoregressive models from multivariate time series is conventionally approached through least squares or maximum likelihood estimation. These methods typically assume a fully connected model which provides no direct insight to the model structure and may lead to highly noisy estimates of the parameters. Because of these limitations, there has been an increasing interest towards methods that produce sparse estimates through penalized regression. However, such methods are computationally intensive and may become prohibitively time-consuming when the number of variables in the model increases. In this paper we adopt an approximate Bayesian approach to the learning problem by combining fractional marginal likelihood and pseudo-likelihood. We propose a novel method, PLVAR, that is both faster and produces more accurate estimates than the state-of-the-art methods based on penalized regression. We prove the consistency of the PLVAR estimator and demonstrate the attractive performance of the method on both simulated and real-world data.

show abstract

Objective Bayesian Edge Screening and Structure Selection for Ising Networks

et al. 2022

View full text Add to dashboard Cite

The Ising model is one of the most widely analyzed graphical models in network psychometrics. However, popular approaches to parameter estimation and structure selection for the Ising model cannot naturally express uncertainty about the estimated parameters or selected structures. To address this issue, this paper offers an objective Bayesian approach to parameter estimation and structure selection for the Ising model. Our methods build on a continuous spike-and-slab approach. We show that our methods consistently select the correct structure and provide a new objective method to set the spike-and-slab hyperparameters. To circumvent the exploration of the complete structure space, which is too large in practical situations, we propose a novel approach that first screens for promising edges and then only explore the space instantiated by these edges. We apply our proposed methods to estimate the network of depression and alcohol use disorder symptoms from symptom scores of over 26,000 subjects.

show abstract

Marginal Pseudo-Likelihood Learning of Discrete Markov Network Structures

Cited by 24 publications

References 22 publications

Structure Learning of Contextual Markov Networks using Marginal Pseudo‐likelihood

Structure Learning of Contextual Markov Networks using Marginal Pseudo‐likelihood

High-dimensional structure learning of sparse vector autoregressive models using fractional marginal pseudo-likelihood

Objective Bayesian Edge Screening and Structure Selection for Ising Networks

Contact Info

Product

Resources

About