Stochastic optimization methods for structure learning in Gaussian graphical models and the general log-determinant optimization

Abstract

Graphical models compactly represent the most significant interactions of multivariate probability distributions, provide an efficient inference framework to answer challenging statistical queries, and incorporate both expert knowledge with data to extract information from complex systems. When the graphical model is assumed to be Gaussian, the resulting model features attractive properties and appears frequently in cutting-edge applications. In the application of Gaussian graphical models, it is fundamental and challenging to learn the graph structure from given multivariate data. The time complexity of existing methods is subject to the cumbersome operations such as matrix inversion, full spectral decomposition, and Cholesky decompositions, which generally take O(N 3 ) arithmetic operations. This computational bottleneck severely impedes the scalability of those methods into problems at large scale. Moreover, structure learning in Gaussian graphical model is a specialized case of the general log-determinant optimization, which entails general matrix constraints and covers much broader fields of applications. The bottleneck of the cubic time complexity also occurs in existing methods for the general log-determinant optimization. In this thesis, we propose a novel method to learn the structure of the Gaussian graphical model with the aid of stochastic approximation so that the time complexity can be reduced from O(N 3 ) to O(N 2 ). In addition, we generalize the proposed method within the framework of primal-dual optimization so that we can settle the general log-determinant optimization problem with similar efficiency.