On optimum strategies for minimizing the exponential moments of a loss function

Abstract

Abstract. We consider a general problem of finding a strategy that minimizes the exponential moment of a given cost function, with an emphasis on its relation to the more common criterion of minimization the expectation of the first moment of the same cost function. In particular, the basic observation that we make and use is about simple sufficient conditions for a strategy to be optimum in the exponential moment sense. This observation may be useful in various situations, and application examples are given. We also examine the asymptotic regime and investigate universal asymptotically optimum strategies in light of the aforementioned sufficient conditions. Finally, we propose a new route for deriving lower bounds on exponential moments of certain cost functions (like the square error in estimation problems) on the basis of well known lower bounds on their expectations.

On optimum strategies for minimizing the exponential moments of a loss function

“…To see why this is true, observe that for large α, N (0, 1/(2α)). Thus, for D ≤ 1/(2α), the rate-distortion function of Q agrees with the Shannon lower bound (see [14]), R L (D) = log M + 1 2 log(1/[2αD]), which, for D = 1/(2α), gives a coding rate of log M , just like the aforementioned uniform quantizer.…”

“…Then, (x, s * ) = ln |T Q | + k log n ≈ nĤ(x) + k 2 log n, whereĤ(x) is the empirical entropy pertaining to x, and where the approximate inequality is easily obtained by the Stirling approximation. Then, as can be seen in [14]: ln E P θ exp{α (X, s)} ≤ nαH 1/(1+α) (P θ )+α k 2 log n. Rissanen's result is now obtained a special case by dividing both sides by α and taking the limit α → 0.…”

“…The proof appears in the full version of this paper [14]. Partially related results have appeared in the literature of optimization and control (cf.…”

“…Considering now the exponential moment criterion, as a lower bound, we have (see [14] for a full proof):…”

“…Q, becomes E Q { f (U ) 2 P (U )/Q(U )} ≤ nΓ, but "power" function f (u) 2 P (u)/Q(u), with P and Q being Gaussian densities, is no longer the usual quadratic function of f (u) for which there is a linear encoder and decoder that is optimum. More details on the optimum encoder and decoder can be found in [14].…”