A comparison of related density-based minimum divergence estimators

Jones, M. C.; Hjort, Nils Lid; Harris, Ian R.; Basu, Ayanendranath

doi:10.1093/biomet/88.3.865

Cited by 114 publications

(143 citation statements)

References 16 publications

Supporting

Mentioning

142

Contrasting

Order By: Relevance

“…For example, if the tuning parameter γ is large, then the robustness of the Gamma divergence will be strong but the efficiency could be lower, and vice versa, if the tuning parameter has small positive value, then the robustness of the method would be not strong but the efficiency will be higher [35]. The relation between the efficiency and the tuning parameters was discussed by several authors (see Jones et al [85], Basu et al [67], and Fujisawa and Eguchi [35]). …”

Section: Conclusion and Discussionmentioning

confidence: 99%

Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities

Cichocki

Амари

2010

Entropy

381

313

View full text Add to dashboard Cite

In this paper, we extend and overview wide families of Alpha-, Beta-and Gamma-divergences and discuss their fundamental properties. In literature usually only one single asymmetric (Alpha, Beta or Gamma) divergence is considered. We show in this paper that there exist families of such divergences with the same consistent properties. Moreover, we establish links and correspondences among these divergences by applying suitable nonlinear transformations. For example, we can generate the Beta-divergences directly from Alpha-divergences and vice versa. Furthermore, we show that a new wide class of Gamma-divergences can be generated not only from the family of Beta-divergences but also from a family of Alpha-divergences. The paper bridges these divergences and shows also their links to Tsallis and Rényi entropies. Most of these divergences have a natural information theoretic interpretation.

show abstract

Section: Conclusion and Discussionmentioning

confidence: 99%

Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities

Cichocki

Амари

2010

Entropy

381

313

View full text Add to dashboard Cite

show abstract

“…In the context of probability distributions, these weighting factors may control the influence of likelihood ratios p it /q it . It has been shown [55] and [56] that the parameter β determines a tradeoff between robustness to outliers (for β > 0) and efficiency (for β near 0). In the special case of β = 1 the Euclidean distance is obtained, which is known to be more robust and less efficient than the Kullback-Leibler divergence β = 0.…”

Section: Why Is Ab-divergence Potentially Robust?mentioning

confidence: 99%

Generalized Alpha-Beta Divergences and Their Application to Robust Nonnegative Matrix Factorization

Cichocki

Cruces

Амари

2011

Entropy

175

172

View full text Add to dashboard Cite

We propose a class of multiplicative algorithms for Nonnegative Matrix Factorization (NMF) which are robust with respect to noise and outliers. To achieve this, we formulate a new family generalized divergences referred to as the Alpha-Beta-divergences (AB-divergences), which are parameterized by the two tuning parameters, alpha and beta, and smoothly connect the fundamental Alpha-, Beta-and Gamma-divergences. By adjusting these tuning parameters, we show that a wide range of standard and new divergences can be obtained. The corresponding learning algorithms for NMF are shown to integrate and generalize many existing ones, including the Lee-Seung, ISRA (Image Space Reconstruction Algorithm), EMML (Expectation Maximization Maximum Likelihood), Alpha-NMF, and Beta-NMF. Owing to more degrees of freedom in tuning the parameters, the proposed family of AB-multiplicative NMF algorithms is shown to improve robustness with respect to noise and outliers. The analysis illuminates the links of between AB-divergence and other divergences, especially Gamma-and Itakura-Saito divergences.Entropy 2011, 13 135

show abstract

“…A wide class of parametric minimum divergence estimators may be found in Basu et al (1998), Jones et al (2001). Other diergence-based estimators are found in Menéndez et al (1995Menéndez et al ( ,2001).…”

Section: Divergence-based Inferencementioning

confidence: 99%

A Refined Jensen's Inequality in Hilbert Spaces and Empirical Approximations

Leorato

2008

SSRN Journal

View full text Add to dashboard Cite

Let f : X → R be a convex mapping and X a Hilbert space. In this paper we prove the following refinement of Jensen's inequality:for every A, B such that E(X |X ∈ A ) = E(X |X ∈ B ) and B ⊂ A. Expectations of Hilbert space valued random elements are defined by means of the Pettis integrals. Our result generalizes a result of Karlin and Novikov (1963), who derived it for X = R. The inverse implication is also true if P is an absolutely continuous probability measure. A convexity criterion based on the Jensen-type inequalities follows and we study its asymptotic accuracy when the empirical distribution function based on a n−dimensional sample approximates the unknown distribution function. Some statistical applications are addressed, such as nonparametric estimation and testing for convex regression functions or other functionals.

show abstract

A comparison of related density-based minimum divergence estimators

Cited by 114 publications

References 16 publications

Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities

Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities

Generalized Alpha-Beta Divergences and Their Application to Robust Nonnegative Matrix Factorization

A Refined Jensen's Inequality in Hilbert Spaces and Empirical Approximations

Contact Info

Product

Resources

About