On Learning Parametric Non-Smooth Continuous Distributions

Class-Specific Distribution Alignment for semi-supervised medical image classification

“…In this paper, we tackle the problem of empirical distribution mismatch, which is a key issue in semi-supervised learning (e.g., [21,5,50]). In principle, the labeled and the unlabeled training samples are assumed to be drawn from an identical distribution.…”

Concentration Bounds for Discrete Distribution Estimation in KL Divergence

“…Here (12) follows from the fact that conditioned on N 2i−1 + N 2i , the log (N 2i−1 + 1) + log (N 2i + 1) terms are independent. (13) follows from that all N 2i−1 + N 2i 's have the same distribution and the linearity of sum of expectations.…”

“…Formalizing this heuristic might be an interesting future research direction. By [12], as n → ∞ for the uniform distribution,…”

“…where the minimum is taken over all possible n-sample estimators p : [k] n → ∆ k , and the maximum over all possible discrete distributions on [k]. The precise asymptotic minimax rates (including the leading constant) for a fixed k and as n increases are known for 1 distance [12], 2 distance [14, p. 349], scaled 2 losses and chi-squared type distances [18,19,12], as well as KL divergence [5]. Distribution-specific rates are also known for 1 distance [11,6,7].…”

Smoothed Analysis of Sequential Probability Assignment

“…Sequential probability assignment is a classic topic in information theory with extensive literature, see the survey by Merhav and Feder [1998] and the references within. In particular, the idea of probability assignments that are Bayesian mixtures over the reference class of distributions [Krichevsky and Trofimov, 1981] is of central importance-such mixture probability assignments arise as the optimal solution to several operational information theoretic and statistical problems [Kamath et al, 2015]. It is also known that the Bayesian mixture approach often outperforms the "plug-in" approach of estimating a predictor from the reference class and then playing it [Merhav and Feder, 1998].…”