Entropy expressions for multivariate continuous distributions

Darbellay, Georges A.; Vajda, Igor

doi:10.1109/18.825848

Cited by 72 publications

(53 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We chose four bivariate distributions for which the mutual information is known analytically [10]. These are the Gaussian distribution, the Pareto distribution, the gamma-exponential distribution, and the ordered Weinman exponential distribution.…”

Section: Resultsmentioning

confidence: 99%

Estimation of the information by an adaptive partitioning of the observation space

Darbellay

Vajda

1999

IEEE Trans. Inform. Theory

Self Cite

397

305

View full text Add to dashboard Cite

show abstract

Section: Resultsmentioning

confidence: 99%

Estimation of the information by an adaptive partitioning of the observation space

Darbellay

Vajda

1999

IEEE Trans. Inform. Theory

Self Cite

397

305

View full text Add to dashboard Cite

show abstract

“…., m is the Jacobian of transformation; see, e.g. [9]. Thus, for the continuous case, H (V) can be larger or smaller than, or equal to H (Q) depending on E[log J (V)].…”

Section: Basic Information Measuresmentioning

confidence: 99%

Information measures of Dirichlet distribution with applications

Ebrahimi

Soofi

Zhao

2011

Appl Stoch Models Bus & Ind

View full text Add to dashboard Cite

We explore properties of information measures of the Dirichlet family and related distributions. Representations of the information measures of the Dirichlet family in terms of the information measures of the gamma family reflect the characterization of Dirichlet distribution in terms of the ratios of independent gamma distributions to their sum. We present measures of information provided by a multinomial vector about the multinomial parameters under the Dirichlet prior and measures of information loss due to aggregation of Dirichlet components and the multinomial categories. New information characterizations of the bivariate Dirichlet distribution in terms of survival and hazard gradient constraints are given. The information analysis of contingency tables is briefly described. An example illustrates applications of Dirichlet prior for inferences about information measures of contingency tables and Kendall's tau for ordinal data. Results that compare solicitation of the mean and mode vectors for the Dirichlet model are given and are illustrated through an example from a study of business executives' environmental uncertainty in the aftermath of 9/11 terrorist attacks. The probability integral transformation is proposed for facilitating implementation of the maximum entropy Dirichlet process prior procedure for inference about the fit of continuous distributions to the data.

show abstract

“…In the table that follows we have computed explicitly entropic priors for some common uni variate distributions [10,21]. For some canonical multivariate densities [11] we also have…”

Section: Entropic Priors For Canonical Likelihoodsmentioning

confidence: 99%