Computational Information Geometry in Statistics: Theory and Practice

Abstract

A broad view of the nature and potential of computational information geometry in statistics is offered. This new area suitably extends the manifold-based approach of classical information geometry to a simplicial setting, in order to obtain an operational universal model space. Additional underlying theory and illustrative real examples are presented. In the infinite-dimensional case, challenges inherent in this ambitious overall agenda are highlighted and promising new methodologies indicated.

Geometry of Goodness-of-Fit Testing in High Dimensional Low Sample Size Modelling

“…3 (b) and (c) below. However, as discussed in [6] and [3], these curvature terms grow unboundedly as the boundary of the probability simplex is approached. Since this region plays a key role in modelling in the sparse setting -the MLE often being on the boundary -extensions to the classical theory are needed.…”

The Information Geometry of Sparse Goodness-of-Fit Testing

“…This paper, together with [2], start such a development. This work is related to similar ideas in categorical, (hierarchical) log-linear, and graphical models [1,[11][12][13]. As stated in [13], "their statistical properties under sparse settings are still very poorly understood.…”

“…Geometric terms are used to correct for skewness and other higher order moment (cumulant) issues in the sampling distributions. However, these correction terms grow very large near the boundary [1,10]. Since this region plays a key role in modelling in the sparse setting-the maximum likelihood estimator (MLE) often being on the boundary-extensions to the classical theory are needed.…”

“…We also use these simulation results to explore currently open research questions. As can be seen, the overarching theme is the importance of working in the geometry of the extended exponential family [1], rather than the traditional manifold-based structure of information geometry.…”

“…Both papers investigate the issue of goodness-of-fit testing in the high dimensional sparse extended multinomial context, using the tools of Computational Information Geometry (CIG) [1]. Section 2 gives formal proofs of two results, Theorems 1 and 2, which were announced in [2].…”