Some Useful Integral Representations for Information-Theoretic Analyses

Abstract

This work is an extension of our earlier article, where a well-known integral representation of the logarithmic function was explored and was accompanied with demonstrations of its usefulness in obtaining compact, easily-calculable, exact formulas for quantities that involve expectations of the logarithm of a positive random variable. Here, in the same spirit, we derive an exact integral representation (in one or two dimensions) of the moment of a nonnegative random variable, or the sum of such independent random variables, where the moment order is a general positive non-integer real (also known as fractional moments). The proposed formula is applied to a variety of examples with an information-theoretic motivation, and it is shown how it facilitates their numerical evaluations. In particular, when applied to the calculation of a moment of the sum of a large number, n, of nonnegative random variables, it is clear that integration over one or two dimensions, as suggested by our proposed integral representation, is significantly easier than the alternative of integrating over n dimensions, as needed in the direct calculation of the desired moment.

Expected Logarithm and Negative Integer Moments of a Noncentral χ2-Distributed Random Variable

“…The recently proposed trick of representing the logarithm by an integral [7] turned out to be very helpful in the proof of the continuity of the expected logarithm (see Appendix A.3). While in general very well behaved, the logarithmic function nevertheless is a fickle beast due to its unboundedness both at zero and infinity.…”

“…Instead we rely on a trick recently presented in [7] that allows us to write the expected logarithm with the help of the MGF:…”

“…So, for example, in the field of information theory, there is a close relationship between the expected logarithm and entropy, and thus the expected logarithm of a noncentral χ 2 -distributed RV plays an important role, e.g., in the description of the capacity of multiple-input, multiple-output noncoherent fading channels [1,2]. Many more examples in the field of information theory can be found in [7].…”