Consideration on Singularities in Learning Theory and the Learning Coefficient

Aoyagi, Miki

doi:10.3390/e15093714

Cited by 4 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many probabilistic models such as mixture models, hidden Markov models, and neural networks are singular. To cope with the problem of the singularities, an analysis method based on algebraic geometry has been proposed [ 7 ], and asymptotic properties of the generalization performance and of the marginal likelihood have been investigated in mixture models [ 8 ], hidden Markov models [ 9 ], neural networks [ 7 , 10 ], etc.…”

Section: Introductionmentioning

confidence: 99%

Hidden Node Detection between Observable Nodes Based on Bayesian Clustering

Yamazaki

Motomura

2019

Entropy

View full text Add to dashboard Cite

Structure learning is one of the main concerns in studies of Bayesian networks. In the present paper, we consider networks consisting of both observable and hidden nodes, and propose a method to investigate the existence of a hidden node between observable nodes, where all nodes are discrete. This corresponds to the model selection problem between the networks with and without the middle hidden node. When the network includes a hidden node, it has been known that there are singularities in the parameter space, and the Fisher information matrix is not positive definite. Then, the many conventional criteria for structure learning based on the Laplace approximation do not work. The proposed method is based on Bayesian clustering, and its asymptotic property justifies the result; the redundant labels are eliminated and the simplest structure is detected even if there are singularities.

show abstract

Section: Introductionmentioning

confidence: 99%

Hidden Node Detection between Observable Nodes Based on Bayesian Clustering

Yamazaki

Motomura

2019

Entropy

View full text Add to dashboard Cite

show abstract

“…The papers [ 20 , 21 ] derived bounds on the learning coefficients for Vandermonde matrix-type singularities and explicit values under some conditions.…”

Section: Introductionmentioning

confidence: 99%

Learning Coefficient of Vandermonde Matrix-Type Singularities in Model Selection

Aoyagi

2019

Entropy

Self Cite

View full text Add to dashboard Cite

In recent years, selecting appropriate learning models has become more important with the increased need to analyze learning systems, and many model selection methods have been developed. The learning coefficient in Bayesian estimation, which serves to measure the learning efficiency in singular learning models, has an important role in several information criteria. The learning coefficient in regular models is known as the dimension of the parameter space over two, while that in singular models is smaller and varies in learning models. The learning coefficient is known mathematically as the log canonical threshold. In this paper, we provide a new rational blowing-up method for obtaining these coefficients. In the application to Vandermonde matrix-type singularities, we show the efficiency of such methods.

show abstract

Learning Coefficients and Reproducing True Probability Functions in Learning Systems

Aoyagi

2017

Trends in Mathematics

View full text Add to dashboard Cite

Consideration on Singularities in Learning Theory and the Learning Coefficient

Cited by 4 publications

References 19 publications

Hidden Node Detection between Observable Nodes Based on Bayesian Clustering

Hidden Node Detection between Observable Nodes Based on Bayesian Clustering

Learning Coefficient of Vandermonde Matrix-Type Singularities in Model Selection

Learning Coefficients and Reproducing True Probability Functions in Learning Systems

Contact Info

Product

Resources

About