Joint range of f-divergences

Abstract

I. DIVERGENCES AND DIVERGENCE STATISTICSMany of the divergence measures used in statistics are of the f -divergence type introduced independently by I. Csiszár [1], T. Morimoto [2], and Ali and Silvey [3]. Such divergence measures have been studied in great detail in [4]. Often one is interested inequalities for one f -divergence in terms of another f -divergence. Such inequalities are for instance needed in order to calculate the relative efficiency of two fdivergences when used for testing goodness of fit but there are many other applications. In this paper we shall study the more general problem of determining the joint range of any pair of f -divergences. The results are useful in determining general conditions under which information divergence is a more efficient statistic for testing goodness of fit than another f -divergence, but will not be discussed in this short paper.Lett . Assume that P and Q are absolutely continuous with respect to a measure µ, and that p = dP dµ and q = dQ dµ . For arbitrary distributions P and Q the f -divergence D f (P, Q) ≥ 0 is defined by the formula1) (for details about the definition (1) and properties of the fdivergences, see [5], [4] or [6]). With this definitionExample 1: The function f (t) = |t − 1| defines the L 1 -distancewhich plays an important role in information theory and mathematical statistics (cf.[7] or [8]). In (1) is often taken the convex function f which is one of the power functions φ α of order α ∈ R given in the domain t > 0 by the formulaα(α − 1) = 0 (3) D(P Q) V (P, Q) 2 3 1 0 1 2 Fig. 1. The joint range of total variation V and information D as determined in [8]. It was also proved that any point in the range and by the corresponding limits φ 0 (t) = − ln t + t − 1 and φ 1 (t) = t ln t − t + 1. (4)The φ-divergencesbased on (3) and (4) are usually referred to as power divergences of orders α. For details about the properties of power divergences, see [5] or [6]. Next we mention the best known members of the family of statistics (5), with a reference to the skew symmetry D α (P, Q) = D 1−α (Q, P ) of the power divergences (5).Example 2: The χ 2 -divergence or quadratic divergence D 2 (P, Q) = D −1 (Q, P ) = 1 2 k j=1 (p j − q j ) 2 q j