Moreau-Yosida $f$-divergences

Terjék, Dávid

doi:10.48550/arxiv.2102.13416

Cited by 1 publication

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 1: Common ϕ-divergences and their associated ϕ * ; drawn from [106] which has further examples.…”

Section: Discussionmentioning

confidence: 99%

“…Xu et al [122] proposed a generalization of Shannon Mutual information by restricting the of functions optimised over in a variational representation, motivated slightly differently to the F -information of the present paper -they motivate their definition on computational grounds, and observe as a consequence the estimation performance improves. (Note the brief discussion of F -mutual information in Appendix C.) Terjék [106] regularised the optimisation for binary ϕ divergences with a Wasserstein regulariser. More generally, Birrell et al [16] considered a larger range of F for arbitrary ϕ.…”

Section: Precursors Of F -Informationmentioning

confidence: 99%

“…Such representations have attracted some attention recently; for example [2, section 4.3] and[106,91,17]. A focus of these works is to develop restrictions on the class of functions one optimises over in order to aid their statistical estimation; the F -information of the present paper can be seen to embrace a similar philosophy.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Information Processing Equalities and the Information-Risk Bridge

Williamson¹,

Cranko²

2022

Preprint

View full text Add to dashboard Cite

We introduce two new classes of measures of information for statistical experiments which generalise and subsume ϕ-divergences, integral probability metrics, N-distances (MMD), and ( f , Γ) divergences between two or more distributions. This enables us to derive a simple geometrical relationship between measures of information and the Bayes risk of a statistical decision problem, thus extending the variational ϕ-divergence representation to multiple distributions in an entirely symmetric manner. The new families of divergence are closed under the action of Markov operators which yields an information processing equality which is a refinement and generalisation of the classical data processing inequality. This equality gives insight into the significance of the choice of the hypothesis class in classical risk minimization.

show abstract

“…Table 1: Common ϕ-divergences and their associated ϕ * ; drawn from [106] which has further examples.…”

Section: Discussionmentioning

confidence: 99%

Section: Precursors Of F -Informationmentioning

confidence: 99%