Reproducing Distributions for Machine Learning

Abstract

A model is proposed for learning the nature and value of an unknown parameter, or unknown parameters, in a probability distribution which forms part of a body of statistics related to some system or process.The model is Bayesian, involving the assumption of an a priori probability distribution over the possible values of the unknown parameters; the performance of experiments to gain information about the parameters; and the alteration of the a priori probabilities by Bayes' rule.In the limit, as the number of experiments approaches infinity, the a posteriori distribution in most cases encountered in practice approaches a delta function at the true values of the unknown parameters, so the system learns the values of the parameters exactly.