Further advances on Bayesian Ying-Yang harmony learning

Abstract

After a short tutorial on the fundamentals of Bayes approaches and Bayesian Ying-Yang (BYY) harmony learning, this paper introduces new progresses. A generic information harmonising dynamics of BYY harmony learning is proposed with the help of a Lagrange variety preservation principle, which provides Lagrange-like implementations of Ying-Yang alternative nonlocal search for various learning tasks and unifies attention, detection, problem-solving, adaptation, learning and model selection from an information harmonising perspective. In this framework, new algorithms are developed to implement Ying-Yang alternative nonlocal search for learning Gaussian mixture and several typical exemplars of linear matrix system, including factor analysis (FA), mixture of local FA, binary FA, nonGaussian FA, de-noised Gaussian mixture, sparse multivariate regression, temporal FA and temporal binary FA, as well as a generalised bilinear matrix system that covers not only these linear models but also manifold learning, gene regulatory networks and the generalised linear mixed model. These algorithms are featured with a favourable nature of automatic model selection and a unified formulation in performing unsupervised learning and semi-supervised learning. Also, we propose a principle of preserving multiple convex combinations, which leads alternative search algorithms. Finally, we provide a chronological outline of the history of BYY learning studies.

Bi-linear matrix-variate analyses, integrative hypothesis tests, and case-control studies

“…This bi-linear form leads us to matrix-variate LDA and factor analyses in (Xu 2013a(Xu , 2013b. Also, using matrix normal distribution, the implementations are made by the Bayesian Ying Yang harmony learning (Xu 1995(Xu , 2015.…”

“…as the matrix-variate counterpart of Equation (24), where parameters are typically estimated by the maximum likelihood principle (Xu, 2015). Generally, with help of Equation (25), we may also develop statistics for distributions other than matrix normal distributions.…”

“…Firstly, proposed in (Xu 1995) and systematically developed in the past two decades, the BYY harmony learning on typical structures leads to new model selection criteria, new techniques for implementing learning regularisation, and developing a class of algorithms that implement automatic model selection during parameter learning. Readers are referred to (Xu 2010(Xu , 2012b(Xu , 2015 for the latest introduction about the BYY harmony learning.…”

“…It follows from p(0|x t ) + p(1|x t ) = 1 that we get: For q(x|θ) = G(x|c, ), we implement the maximisation of H(p||q) to estimate θ ω by directly adopting the semi-supervised BYY harmony learning for Gaussian mixture given in (Xu 2015), i.e. its algorithm 9, by which the performances of task A, task B, and task C are coordinated.…”

“…Moreover, H(p||q) can be extended into its matrix-variate counterpart. Particularly, algorithm 9 in (Xu 2015) can be extended into the algorithm 1 given below for learning α ω N X|C x ω , x ω , x ω .…”