A Characterization of the Domain of Beta-Divergence and Its Connection to Bregman Variational Model

Abstract

Abstract:In image and signal processing, the beta-divergence is well known as a similarity measure between two positive objects. However, it is unclear whether or not the distance-like structure of beta-divergence is preserved, if we extend the domain of the beta-divergence to the negative region. In this article, we study the domain of the beta-divergence and its connection to the Bregman-divergence associated with the convex function of Legendre type. In fact, we show that the domain of beta-divergence (and the corresponding Bregman-divergence) include negative region under the mild condition on the beta value. Additionally, through the relation between the beta-divergence and the Bregman-divergence, we can reformulate various variational models appearing in image processing problems into a unified framework, namely the Bregman variational model. This model has a strong advantage compared to the beta-divergence-based model due to the dual structure of the Bregman-divergence. As an example, we demonstrate how we can build up a convex reformulated variational model with a negative domain for the classic nonconvex problem, which usually appears in synthetic aperture radar image processing problems.

Poisson noisy image restoration via overlapping group sparse and nonconvex second-order total variation priors

“…Since our model fits the framework of Eq (16), it remains only to check S1-S5 in Lemma 1. It is easy to verify that D KL (x 1 ||g) is coercive [58] as well as ϕ O (x 2 ) and kr 2 x 3 k p p . Thus, S1 holds.…”

The Bregman-Tweedie Classification Model

“…That is, the convex function of Legendre type (the indefinite integral of the extended exponential function with the reduced domain) and the Bregman-Tweedie divergence. Also, the domain and the range of the various extended exponential related functions are carefully analyzed with the category of the real line R in (1) (Woo, 2017).…”

“…Interestingly, two decades ago, (Lafferty, 1999) had suggested Bregman-beta divergence, which is induced from the beta-divergence, to study the generalized boosting model satisfying the additive structure of the adaboosting. As noticed in (Woo, 2017), the additive structure of the generalized boosting model is well-defined when the base function of the Bregman-beta divergence is a convex function of Legendre type (Rockafellar, 1970;Bauschke and Borwein, 1997). Inspired from (Lafferty, 1999), in this article, we study an extended exponential function (i.e., the generalized exponential function with additional scaling parameter) and fully characterize its domain structure.…”

“…Moreover, we propose the Bregman-Tweedie classification model, the loss function of it is the regular Legendre transformation of the Bregman-Tweedie divergence which is a dual formulation of the Bregman-beta divergence. For more details on these divergences, see (Woo, 2017.…”

“…Z is a set of integer and Z + = {0, 1, 2, ...}. The following categorization (Woo, 2017) of real line R is indispensable while clarifying the domain and the range of the extended exponential function and the high-level mathematical structures, including the Bregman-Tweedie classification model.…”