An estimator of the mutual information based on a criterion for conditional independence

Darbellay, Georges A.

doi:10.1016/s0167-9473(99)00020-1

Cited by 70 publications

(34 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…It has also been found to be robust due to its insensitivity to noise and data transformations (Battiti, 1994;Darbellay, 1999;Soofi & Retzer, 2003). Consequently, MI has recently been found to be a more suitable measure of dependence for IVS during ANN development.…”

Section: Mutual Informationmentioning

confidence: 99%

Review of Input Variable Selection Methods for Artificial Neural Networks

May¹,

Dandy²,

Maier³

2011

Artificial Neural Networks - Methodological Advances and Biomedical Applications

247

163

View full text Add to dashboard Cite

Section: Mutual Informationmentioning

confidence: 99%

Review of Input Variable Selection Methods for Artificial Neural Networks

May¹,

Dandy²,

Maier³

2011

Artificial Neural Networks - Methodological Advances and Biomedical Applications

247

163

View full text Add to dashboard Cite

“…There are two methods for the bivariate histogram approaches such as equidistant cells and equiprobable cells. For more on estimation of mutual information, see Darbellay (1999), Darbellay and Vajda (1999), Harrold et al (2001), Chelikani et al (2003), Kraskov et al (2004) and Zhou et al (2005). The following example shows the MI computation using the estimate of bivariate histogram.…”

Section: Sample Correlation Coefficientmentioning

confidence: 99%

An application of mutual information in mathematical statistics education

Yi¹,

Jang²

2015

Journal of the Korean Data and Information Science Society

View full text Add to dashboard Cite

In mathematical statistics education, we can use mutual information as a tool for evaluating the degree of dependency between two random variables. The ordinary correlation coefficient provides information only on linear dependency, not on nonlinear relationship between two random variables if any. In this paper as a measure of the degree of dependency between random variables, we suggest the use of symmetric uncertainty and λ which are defined in terms of mutual information. They can be also considered as generalized correlation coefficients for both linear and non-linear dependence of random variables.

show abstract

“…From the d-dimensional definition of mutual information given in Equation (6) and applying the change of variables given in Equation (20) we have…”

Section: Of a D-dimensional Random Vector And Entropy Of The Linkagementioning

confidence: 99%

“…Optimized estimators that use adaptive bin sizes have been developed in [6,7]. A very good estimator based on k-nearest neighbor statistics is proposed in [8].…”

Section: Introductionmentioning

confidence: 99%

Non–Parametric Estimation of Mutual Information through the Entropy of the Linkage

Giraudo

Sacerdote

Sirovich

2013

Entropy

View full text Add to dashboard Cite

A new, non-parametric and binless estimator for the mutual information of a d-dimensional random vector is proposed. First of all, an equation that links the mutual information to the entropy of a suitable random vector with uniformly distributed components is deduced. When d = 2 this equation reduces to the well known connection between mutual information and entropy of the copula function associated to the original random variables. Hence, the problem of estimating the mutual information of the original random vector is reduced to the estimation of the entropy of a random vector obtained through a multidimensional transformation. The estimator we propose is a two-step method: first estimate the transformation and obtain the transformed sample, then estimate its entropy. The properties of the new estimator are discussed through simulation examples and its performances are compared to those of the best estimators in the literature. The precision of the estimator converges to values of the same order of magnitude of the best estimator tested. However, the new estimator is unbiased even for larger dimensions and smaller sample sizes, while the other tested estimators show a bias in these cases.

show abstract

An estimator of the mutual information based on a criterion for conditional independence

Cited by 70 publications

References 8 publications

Review of Input Variable Selection Methods for Artificial Neural Networks

Review of Input Variable Selection Methods for Artificial Neural Networks

An application of mutual information in mathematical statistics education

Non–Parametric Estimation of Mutual Information through the Entropy of the Linkage

Contact Info

Product

Resources

About