We give a categorical treatment, in the spirit of Baez and Fritz, of relative entropy for probability distributions defined on standard Borel spaces. We define a category called SbStat suitable for reasoning about statistical inference on standard Borel spaces. We define relative entropy as a functor into Lawvere's category [0, ∞] and we show convexity, lower semicontinuity and uniqueness.Recently there have been some exciting developments that bring some categorical insights to probability theory and specifically to learning theory. These are reported in some recent papers by Clerc, Dahlqvist, Danos and Garnier [DG15, DDG16, CDDG17]. The first of these papers showed how to view the Dirichlet distribution as a natural transformation thus opening the way to an understanding of higher-order probabilities, while the second gave a powerful framework for constructing several natural transformations. In [DG15] the hope was expressed that one could use these ideas to understand Bayesian inversion, a core concept in machine learning. In [CDDG17] this was realized in a remarkably novel way. These papers carry out their investigations in the setting of standard Borel spaces and are based on the Giry monad [Gir81, Law64].
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.